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ABSTRACT 

This paper addresses the problem of building the 
capacity for continuous renewal and reform in the educational system. 
The authors contend that the paradigm of R&D previously used has been 
too narrow, and that it is based on restrictive assumptions about how 
to help schools solve problems. They feel that any revised concept of 
"the R&D system" should include (1) how and by whom problems are 
formulated, (2) a range of likely resources for solving them, and (3) 
the organizational life of operating systems which will affect the 
possibility of implanting a solution. The document first presents 
some history of the R&D system, from the Cooperative Research Act 
through recent construction of new facilities and the letting of 
multi-year contracts to regional educational laboratories. An overall 
framework is presented to make the case for stimulating the "R&D way 
of thinking" and refreshing the "user way of thinking" within the 
system. The authors opt for reinforcing the existing research, 
development^ and information system to make its problemsolving less 
dependent on federal whim; and its solutions more possible, 
economical, valid, and communicated rapidly to those in need. They 
describe the needs of an make specific recommendations for operating 
agencies to strengthen their ability to set goals, identify 
achievement barriers, call on resources for solutions, and 
direct ionalize analysis results and creative thinking. No substantive 
agency for research and development or school improvement is 
presented, and the policy and program recommendations with respect to 
implementation apply only to the improved performance of K-12 
schools. (Author/EA) 
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OFFICE OF THE DIRECTOR 



Dear Colleague: 

The enclosed paper describes a particularly important initiative of the 
Institute. In many respects, the success of the NIE rests on our ability 
to stimulate the development of an R&D community that is genuinely useful 
to educators and students, to make research results available to educators, 
and, perhaps most important, to recognize the role of educators not as 
passive recipients of R&D products but rather as innovators for whom research 
is but one resource for a continuous process of renewal and reform* 

Our concern is to learn how to build organizational capacity—-the capacity 
of the R&D community to create information and alternative practices and 
products of value to educators, the capacity of a variety of agencies to 
link research to practice, and the capacity of schools and State agencies 
to engage in a process of continuous improvement that makes the most effec- 
tive use of local resources as well as the products of external R&D. 

The recommendations made in the second half of the paper represent an 
initial response to the problems and opportunities identified in earlier 
sections. Though present budgetary constraints limit severely the invest- 
ment we can make in the proposed activities in the current fiscal year, 
our intention is to devote considerably increased resources to capacity 
building as the funds become available. 

This paper reflects the efforts of the NIE staff, as well as the thinking 
of numerous experts in research, development, publishing, communications, 
knowledge utilization, and institutional change. In addition, we have 
visited practitioners at the State and local levels to elicit their views 
and to observe exemplary renewal and reform programs. As we refine our 
plans and thinking in the coming months and years, we shall continue to 
rely on. views from individuals like yourselves. We look forward to your 
comments and suggestions. 
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INTRODUCTION 

This paper is addressed to the problem of building capacity in the 
educational system for continuous renewal and reform. 

The Education Amendments Act of 1972 declared it to be the policy 
of the United States to 

(i) Help to solve or to alleviate the problems of, and 
promote the reform and renewal of, American education; 

(ii) Advance the practice of education, as an art, science, 
and profession; 

(iii) Strengthen the scientific and technological founda- 
tions of education; and 

(iv) Build an effective educational research and development 
system* 

The Act created the National Institute of Education in order to carry 
out this policy. 

But the listing of these four separate intentions, one after the 
other, without a hint of their "true" causal or even historical order, 
is a nice metaphor for the present situation in our thinking. The Act 
provides no guidance as to the proper balance between long-term research 
to improve the science of education, multi-year development of widely- 
marketable tested products for schools, or more immediate efforts to 
solve or alleviate pressing problems. On which should scarce resources 
be spent, for the "best" results? 

The Office of R&D Resources, assigned to analyze the fourth general 
domain, the R&D system and its improvement, quickly found the complex 
interlocking of all the above policy goals. While the record of sheer 
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growth in the R&D system is astonishing, the task force preparing this 

review was faced also with the necessity of analyzing past policy decisions, 

as well as the aggregate result: 

The apparent choice not to support development of local- 
school-system R&D capacity. 

The growth of targeted R&D towards nationally-defined 
problems, and the resultitlg rapid increase of money 
flowing to non- university, non-profit organizations 
equipped to respond quickly to new possibilities. 

The concomitant decline of basic research funding, at 
least within the Ktlited States Office of Education, 
and the resulting decrease of university faculty 
involved in the R&D system. 

Few resources remaining to be used in relating the 
R&D system to its ultimate clients in the operating 
school system. 

The result of past choices seems to have created a system of external 
agents, new institutions for the most part, anchored neither in any well- 
tested scholarly understanding of a domain of practice, nor in any intimate 
knowledge of operational problems. This may have been inevitable, and does 
not preclude the creation of useful products. It may also have been enor- 
mously important politically, as R&D funds grew to significant size, to 
have specific targets being worked on by contractors who could be held to 
performance standards spelled out by the Government. The amorphousness 
of traditional research, not to mention the likelihood of its irrelevance 
in the short-run, are stiff barriers to political success. However, the 
result, in relation to the reform of education, seems to have been weak; 
an R&D system with growing resources flowing to it, does not seem to have 
articulated well with the operating schools and school districts of the 
nation. 
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Thus, we have not felt it adequate to analyze only within the confines 
of the existing institutions or to suggest solely rearrangements in those 
terms. A commentator looking at the system on the eve of NIE's creation 
said: 

"The whole paradigms of R&D thinking have been firmly 
implanted, the institutional domains that participate 
in R&D are securely established...." 

He saw no reason to predict any change in the second decade of Federal 
support fOiT education R&D. In our view, however, it is precisely the 
"paradigms" that need rethinking, and a fresh balance of support for 
activities sought. This report is intended to both review the existing 
accomplishments, and to support recommendations for redirection^ 

To foreshadow later chapters here, we want to suggest that the paradigm 
of R&D used in past policy has been too narrow, and based on restrictive 
assumptions about how to help schools solve problems. Further, we feel a 
revised concept of "the R&D. system" must include attention to how and by 
whom problems get formulated in the first place; to what might be a range 
of likely resources for solving them, whether through systematic exernal 
development or some other means; and to the organizational life of operat- 
ing systems which will affect the possibility of implanting a solution 
to a problem. 

The chapters that follow give some history of the growth of the 
narrowly^def ined R&D system, from the first steps of the Cooperative 
Research Act. through the massive construction of new facilities and the 
letting of multi-year contracts to Regional Educational Laboratories in 
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recent years. In the service of our argument about the need for balance, 
however, we then describe the nature of the operating educational system. 
The intent is to highlight the variety of actors involved, their varied 
interrelationships, and their varied difficulties needing systematic 
analysis. 

To integrate the data on these two somewhat independent sys**ems, we 
then present an overall framework to make the case for an "R&D way of 
thinking,** which must be stimulated within the operating system, and a 
"user way of thinking" which must in turn be refreshed within the research 
and development system. We suggest that research and development must be 
seen as a general approach to the world, a way of organizational life 
distinct from much current organization functioning in a wide range of 
institutions. In the specific recommendations which follow, the overall 
goal is to reinforce such a general approach wherever possible. This 
means reinforcing the existing research, development, and information 
system so that its ways of finding problems to work on are made less 
dependent on Federal whim; so that its solutions are possible, economical, 
and valid; and so that they are communicated rapidly to those in need. 
From our sense of current styles in operating agencies, we in turn draw 
a picture of needs to strengthen their ability to set goals, identify 
barriers to achievement, call upon resources for solutions (whether 
outside developers or not), and put firmly in place the fruits of analysis 
and creative thinking from all their sources. In each of these aspects of 
our redefined system, we have suggestions for specific study, demonstration, 
or project support activities. 
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Two points need to be made concerning topics this report will not 
address. First, we will not present a substantive agenda for research, 
development; or school improvement. The question of what changes should 
be made in American education and what research and development should 
be supported to facilitate those changes is addressed in other Institute 
papers — our concern here is to develop institutional capacity to carry 
out that program of R&D and implement effectively worthwhile innovations. 

Second, although the Institute's mandate encompasses formal and 
informal education at every level, our policy and program recommendations 
with respect to implementation have as their primary goal the improved 
performance of elementary and secondary schools. We are aware that 
formal schooling constitutes just part — some say a very small part — of 
a child's total educational experience; we also recognize that our focus 
on elementary and secondary schools eliminates vast numbers of influential 
educational institutions: colleges and universities, technical institutes, 
Headstart Centers, alternative and free schools, proprietary schools, 
industrial and corporate training centers, etc. Given our limited 
resources, however, we have decided to place highest priority initially 
on grades K-12 in the public schools for two practical reasons: most 
of the Institute's R&D program is focussed on the public schools and 
what little we know about implementation of education research and 
development is derived from work with public schools. As the Institute 
broadens its program to include other institutional and non-institutional 
elements of the society, we will broaden our inquiry to address questions 
of broader implementation accordingly. 
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If NIE is to be as rational in its own affairs as we would wish the 
thousands of local schools to be, we must not simply assume that the 
problem-as-given is the right one. We wish to stress again that despite 
the notable achievements of the past decade of research, product develop- 
ment, evaluation, and dissemination, the schools are much as they have 
always been, and not always by choice. We attribute this in part to the 
whole way R&D has been thought about—as an external activity whose results 
will inevitably be found useful by operating agencies. However true this 
may be in the hierarchy of the Department of Defense, where innovations can 
be passed without question down the chain of command, experience has taught 
us it is not possible in education. Improvement of the overall systematic 
problem-solving capacity of education requires a fresh look at the 
operating system, and support of its own internal activities towards renewal, 
as well as encouragement of the private market, foundation, and Federal 
R&D response to the schools. Not to redress the balance of emphasis will 
be to continue a poorly conceived paradigm when we have the knowledge to 
revise it. 
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I. THE EVOLVING FEDERAL ROLE IN 
EDUCATION RESEARCH AND DEVELOPMENT 

The history of education research includes periods of vigorous scholar- 
ship and inquiry. In the hundred years prior to 1940, names such as 
Henry Barnard, William James, John Dewey, and Edward L. Thorndike came to 
be as. eminent in education as those in the pantheon of any other field. 
However, the fifteen or twenty years prior to 1955 was a period of decline. 
Education research and development activities were largely conducted by 
faculties of schools of education, supported by university general funds and 
modest assistance from private foundations. The performers operated in 
virtual isolation from the other academic disciplines and, therefore, from 
much of the intellectual leadership in the social sciences. Partly as a 
consequence of this isolation, education research and development in the 
thirties and forties was generally lacking in vigor and intellectual power 
and had little effect on the practice of education. 

THE ADVENT OF FEDERAL SUPPORT 

The critical role played by the research and development (R&D) commu- 
nity in World War II led the Federal Government, in the decade following 
1945, to assume support of research and development activities as a matter 
of national policy. This role is reflected in the rapid growth of all 
Federal R&D expenditures since 1940: 



ERIC 



-8- 



1940 
1948 
1950 
1953 
1958 
1962 
1968 
1973 



$ .07 billion 



86 billion 
1 billion 
3 billion 
5 billion 
10 billion 
17 billion 
17 billion 



By far the largest part of this expenditure has been related to national 
security. Successes in defense and space produced a growing faith in 
R&D as an effective instrument of technological progress, in turn lead- 
ing to its application to various areas of social concern. Between 1963 
and 1972 the "civilian" portion of the Federal research and development 
budget increased from 14% to 27%. 

The education sector's participation in this growth began in 1954, 
with the passage of the Cooperative Research Act, authorizing the Office 
of Education to support education research and related activities, and 
with the initiation of the Course Content Improvement Program, through 
which the National Science Foundation supported curriculum reform in the 
sciences. In 1958, the National Defense Education Act was passed, giving 
the Office of Education authority to support program development in foreign 
languages, guidance, and uses of media and technology. Both CCIP and NDEA . 
were intended to support education in fields considered vital to the 
nation's military and technological strength, but they also established 
important precedents for Federal support of mission-oriented inquiry into 
other educational problems. In succeeding years, the broad authorization 
of the Cooperative Research Act was further supplemented by new legislation 
focussed on specific educational problems: Foreign Currency Financed 
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Research (1961) ; Research Component of Media Services and Captioned Films 
(1963); Education of the Handicapped Research and Demonstration (1964); 
Vocational Education Research and Training (1965); and Library Research 
and Development (1967). 

By 1963 the Office of Education's annual expenditure on research had 
grown to 14 million dollars, and the NSF Course Content Improvement Program 
had reached 13 million dollars. Some success was achieved in attracting 
the interest of researchers located outside schools of education. The 
nvunber of proposals from university scholars located outside schools of 
education increased four-fold while the number from professional educators 
remained about the same. By 1963, the majority of proposals originated 
with members of the social science disciplines. The research agenda 
continued to be determined largely by the initiatives of individual re- 
searchers rather than those of Government policy makers. Projects tended 
to be quite small, typically one investigator with several graduate 
students. There was little action to build specialized institutional 
capacities, to encourage training of R&D personnel, or to establish links 
between research and practice. 

A CONCERN FOR CAPACITY 

Reviewing the Cooperative Research Program in 1964, the Government 
concluded that in its first few years the program had "stimulated quali- 
tative improvement and quantitative expansion in educational research." 
However, it also noted that (1) "...the results of the projects... 
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did not lead directly enough or quickly enough to observable change and 
desired improvement in educational practice and (2) "...the results of 
small scale project research tended to be fragmented, non-cumulative, 
and inconclusive." 

This review led to several conclusions regarding the conduct of educa- 
tion research and development: 

1. Interdisciplinary teams of personnel—including social 
scientists and practitioners as well as school of educa- 
tion faculty — should be involved. 

2. Attention must be given not only to the production of 
knowledge but to its utilization, indicating a need for 
improved relationships between research, development, 
dissemination, and implementation. 

3. Sustained support on an institutional level was 
necessary to provide continuity and sufficient 
personnel and financial resources for addressing 
major educational probletns. 

These conclusions led to a redirection in Federal policy: the stimu- 
lation and support of new institutional arrangements for education R&D. 

Between 1964 and 1969, the Office of Education funded twenty-one 
university-based research and development centers. Each was intended, in 
the words of the initial announcenient, "...to concentrate human and financial 
resources on a particular problem area in education over an extended period 
of time in an attempt to make significant contributions toward an under- 
standing of, and an improvement of educational practice in, the problem 
area." 
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The ten original centers proposed their own content emphases (e.g#, 
teacher education, cognitive learning, evaluation). The other eleven 
centers were created later to focus on Government-chosen problems (i.e., 
vocational/technical education, early childhood education, and education 
of the handicapped) . Most of these centers had a close relationship to a 
school of education. Though the mandate of the centers included the full 
spectrum of research, development, and dissemination, most concentrated 
initially on research. Over time, many began to allocate significant 
portions of their resources to applied research and prototype development. 

In 1966, twenty additional organizations, called regional educational 
laboratories, were established uy the Office of Education under the authority 
of an amendment to the Cooperative Research Act contained in the Elementary 
and Secondary Education Act of 1965. Policy direction for the laboratories 
was at first neither clear nor consistent. While it was generally understood 
that these organizations would not conduct basic research, it was unclear 
what function, or combination of functions, they were to perform: applied 
research, product development, dissemination, training, or service to schools. 
By 1968, product development had been established as their primary function. 

The Elementary and Secondary Education Act amendments to the Cooperative 
Research Act also authorized funds for research training and for the construc- 
tion of specialized research facilities. By 1970, seven laboratory or center 
facilities had been completed or were under construction, and by 1973, some 
1500 students had graduated from R&D training programs supported at over 100 
institutions . 
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During this same period, the Office of Education began development 
of an informa'tion system to serve both the R&D community and the schools: 
the Educational Resources Information Center (ERIC). Over the years, eighteen 
"clearinghouse" organizations have been established at various universities 
and professional societies to collect, screen, catalogue, index, and abstract 
current education literature in selected subjects. These data are sent to 
a central organization to become part of a computerized data base from which 
several publications and indexes are produced. The system includes document 
reproduction services, computer search capability > and reference services. 
It also produces specialized bibliographies, research reviews, and syntheses of 
the literature in particular fields. By 1973, ERIC had become one of the 
largest information storage and retrieval systems in the world. 

RAPID GROWTH~1965 TO 1972 

On the basis of the above legislative authorities, appropriations 
for R&D in the Office of Education grew rapidly to $130 million in 1972. 
Meanwhile, support for education R&D in other Federal agencies had been 
growing apace. The total Federal effort in 1972 was as follows: 

Federal Obligations for Education 
Research and Development, FY 1972, 
by Agency (Millions). 

Agency Oblig . 

National Science Foundation. . $ 19.5 
Office of Child Development.. 17.2 
Office of Economic Opportunity 10.9 

Office of Education ,. 130.1 

Total Federal $177.7 
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The magnitude of the education R&D effort can be better appreciated by 
comparison to other key indicators. This dollar amount for FY 1972 
represented 3.0% of the $5,988 million overall Federal investment in 
education, and .4% of the $48,600 million expended by local education 
agencies for elementary and secondary education. From another perspective, 
education R6eD represented 1.0% of all Federal research and development 
obligations ($16,802 million). Nevertheless, an investment of $177 
million must be considered substantial. 

The rapid growth of Federal commitment to education R&D culminated 
in 1972 with the establishment of the National Institute of Education as 
a separate Federal agency charged with the conduct of education R&D. The 
Institute's initial program consisted largely of R&D projects transferred 
from the Office of Education and the Office of Economic Opportunity. As it 
began operations, NIE faced the immediate task of integrating these projects 
and^jome new initiatives into a coherent program structure that would ful-- 
f;(ll the mandate expressed in the Education Amendments of 1972. 

The establishment of NIE was expected to inaugurate a new phase in 
the development of Federal initiatives for educational renewal and reform. 
In the next chapter we will attempt to assess the effect of Government 
initiatives during the 1964-72 era and the present status of the system 
to produce and utilize knowledge in education. 
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II. THE NATURE OF THE RSD SYSTEM 

Looking back over the past decade, we can see measurable progress 
in building an education R&D system. Financial support has increased 
appreciably, participants now include personnel from a much broader range 
of fields and disciplines, new institutions have been established, 
and new technologies for the conduct of R&D are emerging (including research 
management, the development process, evaluation theory and methodology, 
and information storage and retrieval). Yet, as NIE began operations, 
there were significant problems to overcome in promoting an R&D 
system with the capacity to carry out activities responsive to crucial 
educational problems. Government policy toward education R&D had 
suffered from a rapid rotation of priorities and wide fluctuations in 
funding from year to year; there seemed to be an overemphasis on activities 
which promised quick payoff; personnel resources still seemed to lack the 
quality and mix required by the complexity of the problems faced; linkage 
roles and relationships among the various specialized institutions were 
largely haphazard; and, most importantly, the Government seemed to lack 
an overall strategy for bringing the capacities of education R&D to 
bear in a manner that resulted in improved educational practices in schools. 

In this chapter, we will attempt to characterize the current status 
of the R&D system: the sponsors and organizational performers, the R&D 
personnel, the nature of the system outputs , the linkage and support systems, 
and the impact of R&D on educational practice. However, it is important 
to note that the data on the R&D system are fragmentary and uneven in 
quality, making assessment of systemic performance and trends extremely 
difficult. 
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RESEARCH AND DEVELOPMENT INSTITUTIONS 

Among the sponsors of education R&D, Federal agencies have dominated 
the scene, with obligations for FY 1973 of $245.3 million. Somewhat less 
than half this amount is administered by NIE. Private foundations contrib- 
uted roughly an additional $14 million. It is difficult to estimate the 
investments made by universities, by State and local education agencies, or 
by private industry* 

Universities 

The universities' primary contribution to the education R&D system 
has been in the area of research. However, a number of important changes 
has been taking place. Either in response to feelings of social concern 
or in pursuit of the Federal dollar, universities have undertaken work in 
development, dissemination, and service well beyond their traditional 
research domain. Increasingly, they not only seek grant support for 
faculty initiated research, but also compete for contract work. 

Specialized centers and institutes have been formed to bring together 

university resources in new combinations requiring full-time non-faculty 

investigators and technicians. As mentioned earlier, these have included 

a number of centers established in response to OE initiatives: 

Eight R&D centers (now funded by NIE on a program 
basis rather than through institutional support). 

Seven centers for early childhood research. 

Two centers for research on the handicapped. 

Fourteen instructional materials centers for the 
handicapped. 
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In addition there are many other centers, some focussed entirely on 
educational problems, and some focussed primarily on another problem 
but having a capacity for work on education. 

University R&D is not, however, without its share of problems. New 
structural arrangements have caused strains on the coordination of expanded 
research functions with the instructional role. The continued dependence 
on shrinking "soft" money sources contributes to the current financial 
crisis in universities. Some critics believe that the scientific community 
has become alienated by Office of Education R&D programs due to the shift 
from general research support to targeted research, the uncertainty of funds 
as targets came in and out of fashion, and the heavy emphasis on development. 

Independent Non-Profit Organizations 

Largely in response to the availability of Federal funds, a large 
number of non-profit organizations has been established in the last two 
decades capable of performing quality education research and development. 
In addition to the regional educational laboratories, the larger non- 
profits include some that have special competence in the behavioral sciences 
and others that specialize in policy studies. In a number of cases, non- 
profits and profit-oriented firms organized originally to provide high 
quality R&D services to the Department of Defense have reorganized and 
diversified and offer significant R&D capability in the social sector, 
including education. The non-profits are not without their problems. 
Working in the entrepreneurial mode, living from contract to contract, 
can be a precarious existence, particularly for a new organization which 
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has been unable to accumulate unrestricted working capital. Financial 
insecurity and the resulting difficulty in offering long term careers 
have led to problems in recruiting quality personnel. Finally, respon- 
siveness to sponsor initiatives may have been achieved at the expense 
of the educational systems which are the ultimate clients for respon- 
siveness to research and development. 
Profit-Oriented Organization s 

A large but undetermined numt^er of small to medium-sized firms 
exists, oriented either to the grants economy of Government procurement 
or to the market economy of sales to schools. The movement toward 
Federally sponsored programs of directed R&D has significantly increased 
the share of Government contract work going to private business. 

Between 1969 and 1971, allocations from the Office of Education to 
higher education institutions increased 8%; allocations to non-profit 
institutions increased 200%. Among firms oriented to school sales, a 
few years ago there was a spate of mergers and acquisitions combining 
publishers and systems-oriented companies in attempts to form new capa- 
bilities for developing instructional materials and systems for the 
schools. While many of these ventures may not have been greatly successful, 
it is probable, however, that instructional materials are being developed 
by private business firms under far more disciplined methods than was 
true a decade ago. This is partly the influence of the systems technology 
developed within other areas of the private sector and partly the influence 
of disciplined development as practiced by the laboratories and centers. 
Reliable estimates of the extent of this activity are not, however, available. 
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School Systems and State Education Agencies 

The capacity for systematic problem-identification and product- or 
solution-development and testing is uncommon in local schools, school 
systems, or even States. Research offices in urban districts are generally 
oriented to the collection of school statistics, although in recent years 
many have expanded to carry out required evaluations of Federally financed 
programs. A small number of districts has begun to carry out disciplined 
R&D activities, particularly in the area of curriculum development, 
modification, and evaluation. Some State Education Agencies have planning, 
research, or evaluation offices that carry out disciplined R&D, generally 
at a modest level. State assessment programs, in many cases patterned 
after the National Assessment of Education Progress, are becoming increas- 
ingly common and sophisticated. However, the extent and magnitude of 
these various activities within State and local education agencies are not 
known. 

R&D Activities 

Information about the relative participation of these performers in 
various types of R&D, the allocation of funds among them, and their potential 
capacity to perform particular R&D functions is very sketchy and unreliable. 
The data we present below — on the mushrooming evaluation industry, the test- 
ing industry, participation in NIE's Fundamental Research Program, and the 
overall distribution of Fiscal Year 1972 NIE funds by performer community — 
may be indicative of broader trends, but may also be very misleading and 
certainly should not be read as a complete picture of the capacities and 
participation of the R&D system in education research. 
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In 1971, HEW alone spent $29.6 million for evaluation research, a 
specialty area created by frequent evaluation requirements accompany- 
ing the proliferation of social legislation in the 60 's. of this amount, 
$12.5 million was spent on education Programs. The distribution of those 
HEW funds by type of performer provides evidence of the importance of the 
non-profit and profit sectors in this particular specialty. 



Type of Performer % of Funds 

Profit organization 45% 

Non-profit organization... 29% 

College or university 21% 

State and local agencies.. 4% 
Individual consultants .... 1% 



Another specialty area of research is education testing, also 
dominated by the profit and non-profit sector. Test publishers define 
testing requirements, develop and standardize tests, and provide 
scoring and feedback services. They play an important role in research 
and development in providing standardized and comparable measures of 

educational phenomena that are essential to the conduct of education R&D 
and the evaluation of education programs. The six largest firms account 
for three-quarters of the total test sales in the country and for probably 
a higher percentage of education test sales. 

A third example of specialization is basic research. In 1973, NIE 
operated a Field Initiated Studies Program of grant support for unsolicited 
proposals, primarily for fundamental research. As expected, the univer- 
sities dominated this particular arena, receiving 85% of total FIS funds. 
However, the allocation of NIE funds for all programs reduces university 
participation in NIE sponsored activities to the status of a close second. 
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% of FIS 


% of all 


Type of Performer 


Funds 


NIE Funds 




0 


6.3 




11.7 


40.8 


College or university... 


82.0 


34.6 


State and local agencies 


1.8 


17.6 




4.5 


.7 



Private business may be increasing its participation in Government- 
financed education R&D, but it tends to specialize in areas like 
evaluation research and is still a relatively minor factor in NIE*s 
program, 

R&D PERSONNEL 

The rapid growth of funds for R&D during the 60 *s significantly 
increased the number of professional personnel in education research. 
As a rough index, the membership of the American Educational Research 
Association has grown from 1,205 in 1955 to 9,952 in 1972. Little 
current information is available, however, on the extent of the total 
pool of professional researchers involved in educationally relevant 
subjects. There is also little information on the backgrounds, characteris- 
tics, functions, and productivity of such R£D personnel, A 1968 study of 
AERA members revealed that 75% of the researchers surveyed listed their 
major field of study as education. This finding was substantiated by a 
1971 study of the educational backgrounds of those who published in 
education related journals. 

Federal emphasis on the support of development activities In educa- 
tion has led to a significant shift away from research personnel and 
toward development specialists. Some sources estimate that in the decade 
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between 1964 and 1974, the numbers of development specialists increased 
from 3% to 45% of the education R&D force while research scholars dropped 
from 96% to 38%. 

The advent of large scale programmatic R&D performed within 
specialized laboratories and centers has also affected the nature of 
R&D personnel. Academic researchers in the classic mold are in a 
minority on the staff of a modern R&D center or laboratory. Their 
efforts must be supplemented by those of subject matter specialists, 
computer experts, media specialists, evaluatcrs, trainers, field 
agents, etc. Thus, the great increase in demand has not been for academic 
researchers but for other kinds of specialists. 

While data are scarce, there is little if any reason to believe 

that shortages of skilled personnel are attributable to lack of supply 

from the colleges and universities. However, there continue to be nagging 

criticisms about the quality of personnel, a problem which may be related 

to the general environment in which R&D is conducted. For example: 

Instability in the funding of education R&D and 
the consequent undermining of the growth of strong, 
independent R&D organizations have negative effects 
on the career decisions of qualified personnel. 

When the "great leap forward" began in 1964, the 
field was almost totally devoid of tested models 
or theories for producing planned change in an 
enterprise as complex, tradition-bound, and fragmented 
as education, and equally devoid of the technologies 
required for emergent activities such as development, 
evaluation, and research management. Thus, it was not 
possible to recruit people with these skills; rather, 
people were needed who could invent the new technology 
or adapt it from that in other fields. 
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Studies of other fields have shown the great importance 
of informal scientific communities in setting standards 
of performance, providing incentives for excellence, 
providing channels of communication that short-circuit 
the lags in the more formal mechanisms, and generally 
socializing new members to the "rules of the game/' 
Such communities were very weak and fragmented ten years 
ago and even now are only beginning to gain some 
strength. Quite possibly, Government policies Have 
mitigated against their development. 

If there has been one dominant target of education 
R&D efforts in the past ten years, it has been the 
concern for education of minority groups and the • 
disadvantaged. This concern is now embodied in 
the legislative charter of NIE. R&D personnel who 
are themselves members of minority groups can be 
expected to have insights into these problems not 
available to others. Yet minority groups are 
generally under-represented in the R&D work force. 

Baseline data and policy studies are badly needed before long-range 

programs for upgrading personnel can be proposed. In the meantime, our 

working hypothesis is that the problem can best be attacked indirectly 

by strengthening and stabilizing the system so as to make careers in 

education R&D attractive and by promoting the development of scientific 

and technical communities and new R&D technologies. 



THE PRODUCTS AND THEIR EFFECTS 

Most research is not designed by scholars for immediate application, 
but rather to contribute to a body of knowledge. Developers may ultimately 
find it useful as a point of departure in constructing products that will 
have short-term utility. The exception would be policy research or 
evaluation research, both of which are carried out specifically to inform 
decision-makers . 
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As noted earlier, many Government-funded development efforts have 
been oriented to the production of new materials and teaching techniques 
apparently based on the assumption that the curriculum was the crucial 
leverage point in affecting the instruction and learning of children. 
Thus the goal was to be a tangible product that could enter the class- 
room in the form in which it was designed and that would shape teacher 
and student behavior and convey appropriate subject matter in predict- 
able ways at a reasonable cost. A great variety of new curricula has 
been developed with Federal funds in the last two decades in the 
physical and social sciences, mathematics > and to a lesser extent, in th 
arts and humanities. 

Partly because curriculum developers recognized that many features 
of the school impeded their attempts to install new curricula properly, 
and partly due to a natural variation in the interest of people who 
chose to engage in Federally funded development, a wide range of 
development products has emerged from disciplined R&D in the last two 
decades. These range from new i>edagogical techniques (the Improving 
Teacher Competencies Program) to new approaches to school organization 
(the Multi-Unit School) and to methods for producing major systemic 
changes in the basic structure of education (the Voucher Program) . 

Unfortunately, little is known about the effects of disciplined 
R&D on practice. Support for major Government-funded research usually 
terminates when findings are published or the developed product enters 
the market. Rarely have funds been provided to trace the patterns of 
implementation for new practices after the first surge of adoptions 
has been reported. 
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What is known is not encouraging. An Office of Education report 
prepared in 1969 for the Organization for Economic Cooperation and 
Development (OECD) attempted to assess the effects on practitioners 
of Government-st>bhsored R&D activity in education. The report concluded 
that ".^.we have not been able to collect very good evidence on the 

impact of specified research and development activities and where that 

evidence has been collected, it has generally tended to demonstrate 

rather low levels of effect." Much the same can also be said of 

innovations generated outside the formal R&D system. According to a 

study conducted by John Goodlad and his associates in 1970; 

.A very subjective but nonetheless general impression 
of those who gathered and those who studied the data 
was that some of the highly recommended and publicized 
innovations of the past decade or so were dimly conceived 
and, at best, partially implemented in the schools.... 
[Teachers and principals] claimed individualization of 
instruction, use of a wide range of instructional materials, 
a sense of purpose, group process, and inductive or discovery 
methods when our records showed little or no evidence of 
them. 

Both reports call attention to the difference between adoption and 
implementation. Most of the data on utilization of education R&D relate 
to counts of textbook sales, participants in workshops, or superintendents' 
reports that their school system has adopted a specified product or 
practice. But, as the quotation above suggests, on the few occasions when 
trained observers have actually visited classrooms to look for changes 
in practice that reflect the nature of the innovation reportedly adopted, 
they have found either a very low level of change or no change at all. 
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Though the empirical reports available on the implementation of R&D are 
few in number and generally several years old, they do tend to agree in 
their findings and confirm the intuitive judgments of many observers. 
They ' certainly suggest that implementation may be a major problem for 
the Institute. 

A concern for increasing the impact of education R&D has fostered 
serious interest: in the concept of linkage and support systems to 
better relate the producers and consumers of education R&D. We turn 
now to a brief look at some of the institutions and mechanisms which 
attempt to carry out this function. 

THE LINKAGE AND SUPPORT SYSTEM 

Studies of knowledge production and utilization in education and 
many other fields have consistently pointed to the need for linking 
mechanisms and institutions to connect the products of R&D with the 
practice world. These studies assume that there are useful products 
and processes being created by the R&D community which must be conmiuni- 
cated to (and often adapted by) users so that the results of research 
and development can be put into use more effectively. The linking 
organization becomes the middleman and translator in this process by 
being knowledgeable about the products of R&D, communicating RiD results 
to practitioners, and often helping with implementation. 

In some sectors, most notably agriculture, there is an extremely 
well developed system of linking and support organizations and 
people. Interestingly, the Federal investment in agricultural linking 
activities, $100 million in 1973, exceeded the Federal contribution to 
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agricultural research and development ($90 million) . The linkage funds 
help support 16,000 extension staff, most of whom are based in the field 
to provide information and technical assistance. There is no comparable 

system to support the nation's educational sector. There is, however, 
a variety of institutions which has assumed some responsibility for 
bridging the research and operating school communities. 

One of the oldest linking agents between education R&D and 
practice is the education publisher. Traditionally, the Government 
has relied on the industry to produce and market those instructional 
systems that have a profit potential, once prototype development and 
evaluation have been completed by the non-profit or university developer. 
The largest American publishing company has less than 10% of the total 
market. There are fifteen to twenty companies of significant size, and 
there are literally hundreds of smaller companies publishing materials for 
the educational market. 

Thus the industry is diverse and highly competitive. On the other 
hand, the small size and fragmentation of the industry limit the amount 
of capital available to each and therefore limit the risk a publisher 
may be willing to face on the production of an unusual product or the 
investment he is able to make in research and development. 

The education publisher's ultimate success or failure depends upon 
the purchase of his product. In 23 states there is a system commonly 
known as adoption or listing, in which a state-wide body selects a 
limited number of texts for use in that state. The selections are made 
known to local districts, and each district chooses from among the 

^publications on the list. Such a system operates conservatively. When 
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a limited list of materials is to be approved for an entire state, mater- 
ials that are too innovative or too different have little chance. 

In 27 states there is a system of materials selection called local 
adoption or open territory. Practices in these states vary widely, but 
the common thread is that the local district selects the materials. When 
local districts are the size of Chicago or New York, the process takes 
on many of the aspects of state adoption. 

Some of these factors are changing. As we noted above, those publishers 
purchased by major industrial corporations have access to a large capital 
base and, in some cases, R&D capacity. Some publishers are beginning to 
sell services, as well as instructional materials. And selection procedures 
at all levels of the operating system are becoming more flexible. Never- 
theless, many of the factors cited above still continue to discourage 
innovation. 

There are problems connected with the articulation between 
developers and publishers. Publishers often get involved only when 
development is complete, therefore lacking the opportunity to help shape 
a product^s development to make it more marketable. Insufficiently 
involved in the distribution process, developers cannot assist effectively 
in providing teacher training and other support services to the schools. 
The system as it presently operates does not provide a means to adequately 
forecast the market potential of a development product either when a 
decision is made to develop or when publishers are invited to distribute. 
Nor does the system provide for description and analysis of the strategies 
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used by developers and publishers to disseminate products, 
with the result that little is known about the costs and effects of 
various strategies. Lastly, it is not clear that the Government's 
policy with respect to the conditions under which copyright may be 
held and royalties paid has produ'jed the most favorable incentives to 
widespread and effective utilization of development products. 

Another form of linkage, the Educational Resource Information 
Center (ERIC), described previously, has played an important role in 
the communication of research results. The total ERIC collection has 
grown from 5,000 documents in 1967 to 130,000 in 1973. Standing orders 
for microfiche copies of the entire collection go to 456 organizations, and 
single orders are placed annually for 50,000 hard copy reports and 150,000 
microfiche. Although ERIC has enjoyed some success in producing an 
information base and in preparing general kinds of information products, 
it is not clear that it is adequately serving the needs of its multiple 
clients, particularly practitioners. Many critics of the ERIC system 
believe it is almost entirely an instrument for university researchers 
and rarely serves the real needs of teachers, administrators, and 
parents. The potential of the ERIC system has also been hindered by a 
limited base of information, an inadequate range of services, under- 
utilization of current technologies, as well as insufficient attention 
to the training of potential clients in its use. New approaches to 
information communication need to be based on the characteristics of 
potential users ond their communication behaviors and information needs. 
These approaches should also provide for user feedback to assist in 
modifying the system. 
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Many other linking roles and organizations have sprung from the 
flurry of education research and innovation in the 60 's and the Federal 
concern that increased support for education research result in 
demonstrable effects upon schools. The initiation of Title III (ESEA) 
supplementary centers, vocational education research coordinating units, 
the resource materials centers and regional media centers for the 
handicapped, the Follow-Through program, laboratories and R&D centers, and 
extension agents were all, at least in part, a response to the concern 
for linking research with practice and sustaining the effects of that 
research. 

In addition to the Federal concern for linkage. State Education 
Agencies have attempted to coordinate the bulk of research informa- 
tion and make it more accessible to users. For example, the BOCES in 
New York State and the Intermediate Districts in Pennsylvania attempt 
to fulfill this role as one of their functions. 

Finally, a broad range of professional support arrangements is 
now evolving which links together teachers, principals, students, 
parents* and the general community to stimulate and provide a setting 
for problem-solving in the schools. These linking strategies, which 
include teacher centers, advisories, technical assistance programs, 
parent advisory groups, and various school based in-service education 
strategies, view the traditional consumers of R&D (teachers, principals, 
parents, etc.) as "designers'* of educational improvements 
as well as the recipients of externally developed reforms. In this 
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conception, it is assumed that unless people in the schools possess the 
capacity to identify and generate solutions for their problems, either 
through the creative adaption of R&D products to meet their local needs 
or through the generation of locally developed initiatives, little in 
the way of beneficial reform will occur. 

The goals of this type of linking activity contrast rather signifi- 
cantly with linking arrangements designed primarily to relate the research 
community to the schools. The former emphasizes a delivery strategy, 
although often providing for interaction between the research and practice 
systems. The latter is based on practitioner desires for self -improve-- 
ment and provides an organizational base and access to resources for 
translating these desires into useful forms of assistance. 

Few efforts have been made to docioraent the nature, extent; or impact 
of the various linking organizations and strategies mentioned above. It 
is important to understand the response of schools to external arrange- 
ments which attempt to implement specific R&D products; the effects of 
internal and external mechanisms which provide professional support to 
sustain self-initiated reforms; as well as the natural processes of 
change which occur as schools attempt to alter their behaviors. At this 
point, little is known about the various types of linking roles, their 
effects on client groups, and their potential for promoting educational 
reform and renewal. 

One thing does appear rather clear at this point: linking and 
support mechanisms as well as each of the other elements of the R&D 
system must be grounded in the educational contexts which they are 



-32- 



attempting to affect. It is the operating system of public elementary 
and secondary schools which provides formal instruction for the vast 
majority of school age children. Whoever presumes to participate in 
the improvement of public education must understand the nature of the 
current system. In the next chapter, we present a brief portrait of 
this vast enterprise. 
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III. THE NATURE OF THE OPERATING SYSTEM 

It is logical, but perhaps misleading, for us to have begun a 
discussion of the knowledge production and utilization system in 
education with a discussion of the structure for research and develop- 
ment. The locus of knowledge utilization — and a good deal of knowledge 
generation — in public education is in the institutions that have legal 
responsibility for education in the United States: the State Education 
Agencies and the schools. 

Education is not like the defense or space sectors of our 
economy: the Federal agency that supports research and development 
in education cannot and should not mandate its use. The decisions as 
to what should be changed and how changes should be made are highly 
decentralized in education, and they take place in a social and 
bureaucratic structure that is highly complex and very far removed 
from the structure and values of the Federal R&D establish- 
ments. 

Research and development will influence practice in the schools 
only to the extent that it reflects the needs, values, and ideas of 
practitioners and to the degree that practitioners have incentives 
to use effectively the ideas and innovations that might help them 
do a better job. 



ERIC 



-34- 



Our initial focus on research and development may, in particular, 
have been misleading to the extent that it placed implicit emphasis on 
formal R&D as the principle engine of change and improvement in educa- 
tion. This is the form of myopia that has marked the R&D establishment 
for some time. Given the decentralization and local autonomy that 
characterize education, the degree to which it is subject to highly 
localized influences and constraints, and its non-technological craft 
character, the primary source of energy for reform and renewal must 
be found within the operating system and in the lay community served 
by that system. Research and development can be a very important 
resource for change and improvement, but not the prime mover. 

For all these reasons, the role of the operating system, and in 
■particular the schools, must be a central focus of our inquiry. To 
provide some basis for defining that role, we will try in this chapter 
to describe some of the more important features of the operating system 
as they relate to its capacity for innovation and reform, or, in the 
language of this report, for knowledge production and utilization. 

To begin with, it may be useful to note the scope of the 
national enterprise and to present some data on the distribution of 
problems, resources, and authority that have a direct bearing on 
relative need and capacity for local innovation and knowledge 
utilization. 

The operating education system is composed of more than 17,000 
local school districts;, education agencies for each of the 50 states 
and outlying territories, and in 26 states more than 1,500 intermediate 
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units for public school administration between the state and local 
levels. In 1972, American taxpayers paid over $51 billion to support 
94,000 public elementary and secondary schools. An instructional 
staff of 2,350,000, including more than 2,100,000 classroom teachers, 
was employed to provide educational services to over 45 million students. 

Although education remains the legal obligation of the states, 
the actual operation of schools generally is delegated to local educa- 
tion agencies (LEA^s), Within certain parameters established by the 
state (which vary considerably from state to state with regard to 
areas such as textbook selection, curriculum mandates, school building 
codes, minimum personnel requirements, and assessment procedures), 
local districts are responsible for location and size of schools, 
procurement, staffing, organization, evaluation of pupil performance, 
and instruction* 

The degree of local autonomy depends on the ways in which the 
various states have organized and financed their educational system. 
In Hawaii, there is a single statewide system. Twelve states have 
school district boundaries coterminous with those of county units, 
whereas in New England and several other states the school districts 
are organized on a town or township basis. Many states are organized 
with a school district serving each of their larger cities and the 
remainder of the state consolidated into regional districts. 
Thirteen of the states have less than 100 operating school districts; 
the nationwide median is 236, ranging from Nebraska's 1,250 districts 
to Nevada's 17. 
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The variation is similar with regard to state financing of 
education. In New Hampshire, 89.8% of public school revenue is from 
local sources, compared with a nation-wide average of 51.2% and 
Alaska's low (excluding Hawaii's unitary system) of 11.7%. In 1972-73, 
14 state governments provided 50% or more of the revenues for public 
education in their states while four states provided less than 20%. 

There is frequently as much variation among school districts 
within a state as there is between the different states. Size is one 
critical variable. In the fall of 1972, less than 1% of the nation's 
school systems enrolled 30% of the student population and 41% of the 
systems had fewer than 300 pupils each. In more vivid terms, the 
chancellor of New York City's schools is responsible for the education 
of more children than are enrolled in 39 of the 50 states. 

Another important variable relates to the local resources 
available to support education in different communities, a factor most 
critical in those states in which the state government contributes a- 
relatively small share of the overall, support for schools. In 
California, for example, per pupil expenditure in 1970-71 ranged 
from $577 in Baldwin Park to $1231 in Beverly Hills. In New Jersey, 
the Deal School District spent $1746 per pupil in 1971-72, 
compared with a low of $691 spent by the Pine Hill School District. 

Data from the 1970 census indicate that more than 63% of the 
nation's elementary and secondary school students — almost two-thirds — 
lives in metropolitan areas. Over 25% of our student population lives 
in central cities; most of these are from low- income families and many 
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score below national norms on standardized achievement tests. Results 
on the 1970-71 reading tests administered as part of the National 
Assessment of Educational Progress show nine-year-old students from 
inner city areas with a median percent of correct answers of only 
56.6, compared with 67.0 for rural students, and 81.9 for children 
from suburban areas. 

Research has shown that urban schools generally differ from 
suburban schools in several other important respects. Studies 
conducted by Howard Becker, Patricia Sexton, and others indicate that 
schools serving children from lower-class families tend to have lower 
morale and performance among teachers and principals than schools 
serving a middle-class clientele. Dropout rates and vandalism are 
higher in cities than in suburban areas. 

This quick sketch of the nation's educational system reveals 
considerable variation in arrangements for governance and finance 
and in the distribution of resources and problems. There is comparable 
variation in the system with respect to organizational capacity to 
engage in systematic renewal and to generate and use knowledge for the 
improvement of the schools. 

THE STATE EDUCATION AGENCY 

Any discussion of the organizational structure for the improve- 
ment of public education must begin at the state level. State agencies 
traditionally have been responsible for: 



"38" 

Regulatory functions such as teacher certification, 
building inspections, minimum curriculum and text- 
book guidelines, and student attendance requirements; 
and 

Operational functions such as running specialized 
schools or providing direct services, including 
vocational rehabilitation and teacher placement. 

Since 1965, however, increasing numbers of SEA's have developed a 

capacity for substantive educational leadership • Among these 

leadership functions are: 

development of long-range planning and needs assessment 
procedures > 

identification and diffusion of successful programs, 

provision of information services and technical assistance > 

provision of professional support services for innovation^ and 

redesign of in-service education programs and revision of 
certification requirements • 

The capability of SEA's to undertake any of these leadership functions 
is dependent on several key variables: relations with the governor 
and legislature, the role of the State. Board of Education, the 
agency's internal leadership, and the available resources. (The 
average number of professional staff members in SEA central offices 
is 191, ranging from a low of 47 in South Dakota to almost 1000 in 
New York State.) 

In some respects, a strengthened leadership role for the SEA's 
implies centralization at the State level of the education policy- 
making function, and in practice, it has certainly made the State 
Agencies important factors in the knowledge production and utiliza- 
tion process. To the degree that they assess needs (with increasingly 
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sophisticated techniques); engage in substantive policy planning; 

establish program priorities; systematically search for, develop, 

test, and disseminate informatjion about improved programs and practices; 

and stimulate innovation through direct technical assistance to 

schools — the State Agencies have a uniquely important role to 

play in the knowledge production and utilization system for education. 

The movement toward this leadership function, however, is less 
than ten years old in all but a few states. Capacities to perform 
its component operations have generally evolved in bits and pieces, 
frequently in response to provisions of many different Federal laws 
providing categorical aid to education. They remain, in many instances, 
only partly developed and often not integrated into a coherent design. 
No study has yet been made of the nature, extent, and quality of the 
State Agencies' organizational arrangements for exercising the leader- 
ship function, nor have the various strategies employed by the states 
to stimulate innovation and improvemtint in the schools been described 
or their relative merits assessed. 

THE LOCAL EDUCATIONAL AGENCY 

We turn now to a discussion of the schools' capacity for 
participation in education's knowledge production and utilization system.. 
We noted earlier that formal R&D is not the only, and perhaps not the 
most important, source of ideas and innovations that the scho.ols use-"^ 
or should use — to improve their effectiveness. The practice community 
is constantly perceiving problems, inventing solutions, and sharing those 
solutions with colleagues. 
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Knowledge generated by the R&D process, then, takes its place alongside 
knowledge generated by the practice community as both are utilized by 
the schools to solve their problems. 

In considering our discussion of the schools • capacity to solve 
problems through the generation and utilization of knowledge, bear in 
mind the data we presented above illustrating the diversity within the 
operating system and the difficulty therefore inherent in making 
generalizations about the process of reform and renewal in the schools. 
The problems of a Philadelphia high school with 5400 students on double 
sessions clearly are different from those of a regional high school in 
Montana serving 300 students from a 400 square-mile geographical area. 
The organizational structure and resources available to cope with those 
problems are equally different. Yet despite the extensive variation among 
schools, school systems, and State departments of education, there are some 
common features that characterize the capacity of the operating system to 
generate and use new knowledge to solve problems. 

To begin with, there seems to be a considerable amount of innovative 
activity in schools across the country. In a recent survey of innovation 
in a diverse sample of school districts, Ronald and Mary Havelock found 
a "ubiquitousness of innovation." Of 353 districts, 346 reported an 

innovation of significant proportions during the 1970-71 school year. These 
findings, the authors conclude, seem to ''run counter to the ideas that many 
have of U.S. educational 'establishment' as frozen in its ways, indiffer- 
ent to change and unresponsive to the needs of students/' While their 
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reliance on superintendents' self-reports of innovation can be faulted — 
and definitions of what constitutes a "significant" innovation vary — 
there is little doubt that "being innovative" is a desired characteris- 
tic for most school systems* 

Yet it is not clear that local innovation is organized and 
supported in a way that leads to sustained improvement. Most local 
innovations are limited in scope and tend not to generalize to other 
populations. Idiosyncratic features of the innovation (e.g., a charismatic 
leader, a special group of students, a unique developmental history behind 
the innovation) frequently have precluded its continuation as part of 
the larger system after initial funding ended or its original staff 
had left. 

Nor is it clear that the reported adoption of innovations generated 
by others — either the schools or the R&D system — leads in fact to the 
genuine implementation of the substance of the innovation. As we noted 

above in our discussion of the effects of R&D on practice* the very 
sketchy evidence we have generally shows that when trained observers 
visit schools that report "adoption" of a given innovation, the observers 
have seen little evidence of its actual implementation. 

One might ask how we can suggest on the one hand that the schools 
are characterized by ubiquitousness of innovation," and on the other 
hand report that little real change may have taken place as a result. 
However, it is precisely this condition that leads some observers to speak of 
"faddism" as characterizing the innovation process in education. 
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INCENTIVES FOR INNOVATION 

At a deeper level of analysis, however, the explanation may lie 
in an examination of the incentives for innovation in the schools. 
In some respects, these incentives may be perverse — that is, they may 
promote only certain types of genuine innovation and lead to the desire 
to produce the appearance but not the reality of others. 

Schools do not exist in a market economy. That is, the schools 
cannot choose to specialize in a particular class of clients and the 
clients cannot freely choose among contending suppliers of educational 
services. The "product" that the schools have to offer is not clearly 
defined; the means of most efficiently or effectively producing that 
product are not known; and the degree to which schools produce that product 
independent of other influences is a matter of conjecture. Further, while 

the schools are trying to make their uncertain contribution to these 
uncertain goals using uncertain means to assist a clientele they did 
not choose and which did not choose them, they operate in a fishbowl 
amidst a wide variety of usually conflicting influences. 

In a report we recently commissioned on this subject, John Pincus 
and Daniel Weiler suggest that these factors may in fact produce a set 
of incentives not wholly supportive of vigorous generation, adoption, 
and implementation of innovations. 

In particular, they suggest that there are virtually no incentives 
to implement innovations that would seriously disturb the bureaucratic 
structure of the school, redistribute authority within that structure, 
change the balance between capital investment and labor investment, or 
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change the market structure of education in ways that would force the 
schools to compete for clients with other providers of educational 
services. Within the limits of these constraints, however, the schools 
have an incentive to be responsive to the demand of their clientele 
(and their own sense of professionalism) and to take advantage of recent 
advances in educational practice and knowledge (provided, of course, that 
they can produce some consensus as to what constitutes an advance). 

This last proviso may be of particular significance. The difficulty 
of achieving political consensus on goals and methods (these being often 
hard to distinguish), the vulnerability of the professional educator to 
criticism from even small groups or determined community opponents, 
and the difficulty of conclusively demonstrating the superiority of 
one approach over another, all combine to produce incentives to engage in 

the rhetoric of innovation ("Our school system is among the leaders in 
the state.") but to stop short of actually implementing innovations that 

might badly alienate significant elements of a very diverse consti- 
tuency ("I don't want you experimenting on my kidsl"). 

Against this set of somewhat gloomy observations one must weigh 
the commitment of most educators to their profession and to the 
children their profession serves. One must also acknowledge a certain 
restlessness within any system that resists the status quo and seeks 
change and improvement as a matter of personal satisfaction. These 
factors — hard as it is to calculate their influence — may be the crucial 
factors as we consider the schools as a market for innovation. 
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THE CAPACITY FOR INNOVATION 

Having assessed the incentives for innovation in the schools, 
we turn to a discussion about organizational capacity and resources 
for renewal, innovation, and knowledge utilization in the operating 
educational system. 

Starting at the most basic level, that of the individual teacher, 
it is clear that in many school districts teachers are provided with 
few supports or resources that enable them to be either thoughtful 
consumers of R&D or agents in a self-generated process of renewal 
and reform. For most teachers, once their student teaching experience 
ends, there are few opportunities to receive substantive feedback on 
their performance or assistance in analyzing theiir problems. They 
have insufficient opportunities and tools for making intelligent choices 
among the range of textbooks, instructional systems, and hardware 
they are continually urged to purchase. And they generally lack the 

time, facilities, and financial resources to adapt commercially-available 
materials to meet their own needs. 

To a large extent, teachers^ lack of opportunity to develop the 
capacity to systematically identify and address their major problems 
is attributable to the ways in which most schools operate. In most 
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schools teachers are isolated in their classrooms, with little time 
and few mechanisms for a total staff to identify common problems and 
collaborate on finding solutions to them. Teachers generally have 
very little role in making decisions regarding school policy or major 
instructional issues, and very little control over the manner in which 
those decisions are implemented, 

A similar lack of capacity for sustaining the process of reform 
and renewal characterizes the next organizational context within the 
operating educational system, that of the school district as a 
whole. Few school systems are organized in ways that allow them 
to anticipate and analyze problems, to search for or generate knowledge 
that might be useful in solving those problems, or to make management 
plans for the utilization of that knowledge. 

The investments school districts make to support their own R&D 
activities are miniscule. A 1969 survey by the Bureau of Social Science 
Research showed that three-fifths of the school districts studied had no 
budget provision for research and development; another 20% reported expendi- 
tures of less than $2,000 a year. A 1965 study by the National Education 
Association showed that once operational programs (e.g., testing, data 
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processing) are eliminated, no school districts in this sample spent 
more than .5% of their budget for offices of research,' planning, and 
evaluation. A telephone survey we conducted in October 1973 revealed 
that research and evaluation offices of the country* s 15 largest urban 
school systems continue to receive meager support. Of the 12 systems 
for which data were received, the largest commitment to a research/ 
evaluation office was .73% of the system's budget, and the smallest 
expenditure was a .02% commitment for research. The average for all 
12 was .16%. 

The formal school system is just one element within the overall 
community* however, and that broad community has resources that could 
be directed toward an ongoing process of educational renewal. Unfor-. 
tunately, this is seldom the case. In most communities, education 
remains the carefully-guarded prerogative of the professionals; there 
is little provision for the productive involvement of parents and 
non-professionals in school decision-making. Despite its membership 
of more than 9 million people in over 43,000 local chapters, Parent- 
Teachers Associations generally do not play important roles in the 
formulation of school district policies or in the determination of 
major instructional matters. 

Yet the community can serve as a principal stimulant for major 
changes the schools would not otherwise consider, either because the 
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coramunity feels the schools are not me^tJiig traditional goals or because 
it redefines in some ways the objectives of education. Thus changes in 
the basic market structure of schools (use of public funds to support 
private schools, for example), or changes to significantly affect the 
resource mix or institutional structure of the schools, may be more 
likely to be initiated by the layman than by the educator. 

Despite this rather negative analysis of the current capacity at 
any of the four levels (individual teacher, school, school district, and 
general community) to sustain a process of educational renewal, there are 
indications that this situation can be changed. For example, over the 
past several years a number of teacher centers, advisories, and extension 
agent programs has been established which show some success in supporting 
the innovative efforts of teachers, introducing them to new materials, 
helping them implement those materials in their classrooms, and assisting 
them generally to devise appropriate solutions for the problems they 
face. 

With regard to schools, it is encouraging to see the development' of 
mini-schools and alternative schools in many communities. These insti- 
tutions often are structured in ways which allow the faculty to 
participate in decision-making, creating mechanisms to identify and 
address the school's major problems. Perhaps most encouraging are the 
examples of some "real schools'' in cities— those without additional funds, 
exotic programs, or extensive pre-opening staff training — in which student 
performance and staff morale are high. It can be done, in spite of all 
Q "le traditional factors that would lead one to expect poor results. 
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Likewise, there are some school districts which have developed 
capacities for problem identification, experimentation, and assessment. 
Some school districts have created an organization development office 
responsible for helping staff members diagnose and devise solutions to 
their problems. Several other school systems have strengthened their 
planning and evaluation offices so that they can provide needed analysis 
and support for personnel throughout the district. Still others have 
developed mechanisms' for systematically collecting, analyzing, and dissemi- 
nating information about new ideas, practices, and procedures that provide 
alternative solutions to problems at many levels in the systems. 

There also are communities in which lay citizens have joined with 
the professionals in a cooperative effort to improve the schools. 
There are examples of influential and sophisticated parent advisory 
councils, community-wide planning groups, citizen-run information 
centers, and student advocacy groups where the productive involvement 
of non-professionals has led to more and better use of resources for 
school improvement in addition to serving as a constructive stimulant 
for change. 

The existence of these successful situations is encouraging. 
They demons trate**chat it is possible for elements of the operating 
educational system to build and sustain a capacity to generate and utilize 
knowledge for self-improvement. However, we presently know very little 
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about how these Institutions operate, and In most cases we must be 
tentative In assessing their effectiveness. We cannot now Identify 
those factors which seem to contribute to their apparent successes; 
nor can we say much about the problems encountered by each Institution 
In the development of their programs. We are unsure as to which elements 
are Idiosyncratic and which ones can be replicated elsewhere. Even if 
we were more confident about the elements that were transferable, we 
lack the necessary knowledge to replicate them effectively. 

As we learn more about the ways in which these relatively new 
institutional arrangements can contribute to the schools' capacity to 
solve the problems they confront, we should also learn much about 
the ways in which formal R&D systems and the related linkage and support 
systems can contribute to the Improvement of public education in the 
schools. The products of the research and development system will be 
used only to the extent that practitioners perceive those products to 
be related to problems they face and to the degree that they have incen- 
tives and capacities to use R&D products and other knowledge products 
to further their objectives. 
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IV. RESOURCES FOR PROBLEM-SOLVING: 
TOWARD AN INTERACTIVE MODEL OF EDUCATIONAL CHANGE 

The preceding survey of the various institutions participating 
in the knowledge production and utilization process notes some of the 
progress that has been made and the problems that remain. We believe 
that Government policy during the period under review has been dominated 
by a seriously deficient conception of the change process. This chapter 
will critique that conception, the ''linear model," and make some pro- 
posals which should lead to a more balanced and interactive 
process. 

THE "LINEAR" VIEW OF THE CHANGE PROCESS 

During the last decade, a great deal of the thinking about research 
and development and its relation to change in education was based on 
the nation's experience in rationalizing the process of invention and 
change in the industrial, military, space, and other primarily "hard 
science" sectors- In this conception of education change, the change 
process begins with, and depends on, the accumulation of a body of basic 
knowledge and theory about the fundamental processes of education. This 
basic research function is carried out by university-abased 
researchers working on problems posed largely by the state-of-the-art of 
their respective disciplines. In this conception, the next stage of the 
process is development: making use of research results to create products 
to solve practical problems. "Products" would include physical objects, 
processes, organizational arrangements, or anything else that can be 
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designed or created. Though many variants exist, the process associated 
with development generally includes the following stages: specification 
of objectives, initial design, prototype construction, several stages of 
testing, final design, large scale field testing, suraraative evaluation, 
and, if appropriate, production. 

When development is complete, this view assumes that a product 
exists which meets the objectives it was designed to meet and is 
available for wide distribution to the consxnner at a relatively low — or 
at least competitive — unit cost. 

This model of the change process projects a rather passive or 
compliant role for the school-based consxnner. In those cases where the 
development products include physical objects— textbooks, for exarapler-*- 
the post-development process entails: (1) informing the practitioner 
that the product is available and has certain effects on its target 
population (sometimes this includes demonstration), (2) offering training 
in the use of the product, and (3) making the product itself available, 
usually through the services of a commercial publisher* The last step 
is unnecessary if the products do not include physical objects. 

This approach to change assures that the schools will adopt and 
faithfully install R&D products in some uniform manner, provided only 
that the R&D community demonstrates that they "work," supplies Some 
training in their proper use, and gets the product to the school. The 
implicit Government policy in this view would be to make R&D products 
available to the schools but, beyond the steps outlined above, do 
nothing to assure that such products were implemented, 
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Among the difficulties with this conceptualization, three seem 
fundamental: 

. As it is a researcher^ s view of the worlds it suffers 
from ttcnnel vision. That is, it sees educational 
practice — or at least innovative educational practice — 
as a simple function of external research, development, 
and linkage. Yet what actually goes on in schools is 
a function of many other influences as well, including 
traditional lore and craft knowledge, the perceptions 
practitioners have of the success or failure of trial 
and error innovation, and the particular needs and 
circumstances of specific schools and communities. 

It assumes that the results of research and the 
products of development maintain their identity 
throughout the process and fully specify what 
happens in the classroom. Yet the real world of 
human behavior is never fully subject to standard- 
ization, and the analogy to hardware production 
quickly breaks down. Knowledge and products flowing 
from research and development become switched, re- 
combined, and transformed as they become involved in 
processes of linkage and utilization, 

. There has keen a tendency to assume a one-to-one 
correspondence between institutions and functions — 
i.e., that universities do research, non-profit and 
profit organizations do development, and that schools 
utilize — and all of the "disciplined inquiry" takes 
place outside the schools. Yet it is clear that 
these various performers often play multiple roles, 

SOME PROPOSALS FOR CONCEPTUAL MODIFICATION 

The last point becomes the basis for moving toward a new conceptu-- 
alization. There is one basic distinction among the institutions concerned 
with educational innovation: some institutions do in fact operate 
schools, while others serve as resources. At the same time, operating 
systems participate in the process of knowledge production and utilization 
(KPficU), The matrix which follows shows the array of institutional per- 
formers on one axis and the various KP&U functions on the other. 
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Production and Utilization in Research 
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Major types and domains of activity can be defined by making analytic 
distinctions on these two axes. We have already introduced the distinction 
between the operating system (LEA's and SEA's) and the external resource 
system (the Federal Government, universities, other non-profits, and profits- 
making corporations). On the horizontal axis there is an important break 
between R&D on the one hand and linkage and utilization on the other. The 
end of the R&D process is usually the point at which some specific product 
emerges, and so this point constitutes a "milestone" in the total process. 
However, at that point there may be a fundamental change in the nature of 
the process, and products may lose their identity as they become switched, 
re-combined, and adapted. 

The cells identified by this process define domains of activity 
which are fundamentally different in important ways. Nevertheless, all 
constitute important elements in an overall strategy for building a 
system of knowledge production and utilization in education. Each of 
these .domains deserves separate comment. 

External R&D 

In many ways this is the most well developed domain. There has been 
a growth and differentiation of specialized institutions, and there -is an 
emerging technology and methodology associated with R&D functions in those 
settings. By far the largest part of the Federal investment in research 
and development (narrowly defined) has gone to this sector. 
Nevertheless, this domain is still in its infancy and has experienced 
a number of difficulties as noted in earlier chapters. 
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In particular, the institutional relationships among the Federal 
Government, universities, non-profit organizations, and industry need to be 
re-examined; ways need to be found to support continued development of the 
technology and methodology of conducting and managing education R&D; 
and ways need to be found to improve the quality of personnel and ensure 
greater minority participation. All of this needs to be done in a manner 
which incorporates new understandings about the nature of the other three 
domains and the interrelationships among all four. A later chapter 
presents a set of program initiatives focussed on those problems. 

External Linkage and Support 

Although linkage activities take place across all levels of the resource 
and operating systems, it is convenient to identify a separate set of initi- 
atives which focus on an external national effort. Among these are the 
operations of a comprehensive information system linking all the partici- 
pants in the resource and operating systems with the external institutions 
involved in delivering products and related technical assistance to the 
schools. As we shall argue below, internal linkage is better understood 
as part of a more comprehensive process of internal problem-solving. 

By definition, external agencies do not use "ultimate" R&D products; 
only operating agencies do. However, external agencies do support inter- 
nal innovations by operating agencies. Indeed, much of the Office of 
Education's support for educational innovation in the past decade (e.g., 
ESEA Title III, the Right-to-Read Program) took this form rather than the 
support of external research and development. To the extent that we 
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are successful in developing new forms of systematic innovation within 

operating agencies (either internal R&D or internal problem-solving), 

we might define new programs for external support of internal innovation. 

Internal R&D 

This cell in the matrix indicates our conviction that elements of 
the operation system — the SEA's and LEA's — should be regarded as legitimate 
performers of education R&D* The nature of their activities generally is 
different from those conducted by agencies within the external resource 
system; there is less basic research, for example, and a far less rigorous 
process of development/field testing/evaluation. 

Nevertheless, we have discovered that there are some SEA and LEA 
activities that produce generalizable knowledge and materials. State 
Agencies play a role in such areas as policy research, the development of 
systematic procedures to identify successful programs, and the design 
of diffusion processes. In addition, some school district personnel 
argue for their own R&D capability on the grounds that external R&D 
agencies frequently are insensitive to or unwilling to address certain 
area.s of immediate concern to school district. 

While we have encountered scattered forms of R&D conducted by 
elements of the operating system, there is little to guide us in determining 
how, and in what domains, SEA's and LEA's can play more effective roles 

in the production of generalized knowledge and materials* Accountability 
to local taxpayers would tend to limit the extent to which SEA's and LEA's 
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can make investments in R&D activities whose results are intended primarily 
for audiences outside their immediate jurisdictions, The monitoring 
system and several of the surveys recommended later in this paper will 
provide us with data on the present and potential capabilities of these 
agencies and should help us reach more definitive conclusions about what, 
if anything, NIE should do to strengthen State and local R&D capacities. 

lea's: Internal Problem Solving 

As defined by the matrix in Figure 1, this domain encompasses thie 
functions of linkage and utilization conducted by LEA's, However, our 
analysis indicates that the role of practitioners must be conceived 
not as passive or unaware consumers but as professional individuals who 
have their own insights, goal structures, and decision roles. Further, 
they seem to view the external R&D community not as a provider of techno- 
logical "fixes," but rather as a source of alternate goals, description 
of problems, and options for dealing with them. 

The preceding section has indicated the . importance of practitioners 
reaching out for new ideas, becoming more sophisticated in their demand 
for particular types of innovations, and assessing critically the extent 
to which any given innovation meets their needs • 

Our aim here is not only to help practitioners become more 
sophisticated consumers but to help build capacities in local school 
districts for ongoing analysis and improvement independent of the schools* 
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use of products developed elsewhere. Among the set of practitioners* skills 
we have in mind are the abilities to diagnose problems accurately, 
to identify and search for resources addressing those problems, to devise 
and test experimentally appropriate responses to the diagnosed problems, 
and to evaluate the effects of those responses. This evaluation would 
then lead to further refinements both in the problem definition and in 
understanding the relative merits of the various alternatives. 

The line between internal R&D and the types of internal problem- 
solving described above is a fuzzy one. In addition to some difference 
in methodology, the chief distinction is the degree to which a genera-- 
lizable solution is sought; research and development seeks generaliz- 
able knowledge and generally useful products, while problem-solving 
is more concerned with the local and unique. 

sea's: Internal Policy, Linkage, and Support 
This domain refers to the linkage and utilization functions of 
State Agencies. Earlier sections have discussed the growing role of SEA^s 
in providing services and support to local innovative activities. State 
departments of education also have the responsibility to assess state- 
wide needs and to design programs of dissemination and diffusion that 
allow school districts to learn about and profit from each other's 
activities. 



ERLC 



-60- 



SUMMARY 

None of these domains can be treated in monolithic terms; each needs 
to be differentiated internally. For example, the roles and activities 
of the different institutions operating in the external resource system 
need to be sorted out more carefully; and the flow of scientific and 
technical information within the R&D community must be understood and 
strengthened — not just the flow of information to practitioners. 

By the same token, although we have described the five domains 
of activity separately, v/e must be careful not to treat them in isola- 
tion; indeed much of our attention" must be on how they interact with each 
other. Since our earlier analysis indicates that the interaction among 
the domains presently is very poor, we suggest the following normative 
considerations for the system which needs to be built: 

Researcher, developer, and user should participate 
in an interactive network to influence activities 
at any other point in the network, 

Incentives should be built into the R&D subsystem 
to make it more responsive to the needs, values, and 
ideas of its clients. 

Institutional and communication links should be 
built between the external resource community and 
the operating sectors to facilitate user access 
to the skills and products of the external agencies, 
and to proyide channels through which those agencies 
can receive feedback from practitioners* 
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RECOMMENDATIONS 
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INTRODUCTION TO THE RECOMMENDATIONS 



Our conceptualization of the knowledge production and utilization 
system and the analyses which preceded it provide both a framework and an 
orientation for the program recommendations. We have suggested that 
there needs to be a better balance among the elements of the system, 
especially with regard to the role of school people as generators as well 
as consumers of knowledge. 

Although we must stress the interactive nature of elements within the 
proposed system, for convenience we have divided the recommendations into 
the following categories: 

Developing a monitoring system within NIE to systematically 
and periodically gather and analyze data about the function- 
ing of the knowledge production and utilization system 
which will provide a base for the continued improvement 
of the system. 

Strengthening the external R&D system by developing an 
understanding of its structure and functions; improving 
technology for conducting R&D; increasing the interface 
between the R&D system, the linkage system, and the 
operating system; and devising strategies to improve NIE's 
management of R&D, 

Building a linkage and support system between the external 
research and development system and the schools, including 
an information system, procedures for delivering develop- 
ment products to the schools, and means of building 
capacities of intermediary organizations that link and 
help schools implement R&D products and provide feedback 
to the formal R&D system. 

Building problem- solving capacity in the operating system 
through the creation of new institutional arrangements in 
the schools and SEA's, development of resource organiza- 
tions outside the schools to assist them in the problem- 
solving process, and strengthening the ability of lay 
members of local communities to contribute to the solution 
of problems in their schools. 
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Chapters V through VIII present the elements of our directed 
research program. They describe the nature and sequence of activities 
we think necessary to provide sufficient data and analyses to determine 
future program directions. We cannot rely solely upon our current sense 
of what is needed, however. We must continually allow our perspective to 
be challenged and broadened by studies that we have not, and might not 
otherwise, consider. Consequently, Chapter IX proposes an ongoing program 
of field-initiated studies whereby support would be provided for undirected 
research in the general area of knowledge production and utilization. 
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V. MONITORING THE SYSTEM 

PROBLEM 

It should be clear from earlier sections of this paper that there 
is a great need for better data concerning the knowledge production and 
utilization system and the operating school system it serves. To be sure, 
information is not entirely lacking, but where it exists it is often out- 
of-date, incompatible with data from other sources, or of questionable 
validity. 

Before we can determine how best to intervene to improve the system, 
we need to develop a much better understanding of system dynamics. The 
basic assiamption is that there are ways to systematically improve the 
efficiency and effectiveness of the knowledge production and utilization 
system and that we are not entirely dependent on occasional flashes of 
insight, A body of knowledge has begun to accumulate concerning R&D systems, 
but primarily in non-education fields. We need to test, verify, and 
revise this knowledge base as it relates to the constraints and contexts 
specific to education R&D. 

Generally we have lacked both the data base and the understanding 
of system dynamics needed for effective, rational policy-making. In 
their absence, policies have been determined on the basis of the resolution 
of cross-pressures, analogies drawn from other fields, and the imagination 
and foresight of a few individuals. 
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Lacking a strong data base, NIE's various publics — practitioners, 
the academic community, the state and Federal power structure, and the 
lay public — have lacked an understanding of the scope, magnitude, and 
nature of education R&D activities. 

An R&D program must be firmly rooted in an understanding of the 
nature and problems of the operating system it seeks to serve. It must 
also have reliable information on the impact of its programs, i.e., a 
"feedback loop" permitting judgments about the impact of NIE initiatives 
on both the external R&D system and the operating system (recognizing 
that the effectiveness of both systems is a function of other factors as 
well) . 

To understand the knowledge production and utilization process, much 
less to develop an Institute policy with respect to supporting widespread 
innovations in' the operating system, we need to know a great deal more 
about the system indicators. Principal indicators might include data on: 
types and numbers of personnel and institutions devoted to the creation, 
production, dissemination, and evaluation of innovations; the financial 
resources committed to such activities and their sources; the facilities 
and equipment existing for these activities or developed to meet specific 
local needs; legal and administrative regulators; and indicators of product 
and practice adoption, adaption, or innovation ♦ 

OBJECTIVES 

The objectives for the program to monitor educational change are 
as follows: 
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To achieve an understanding of the process of knowledge 
production and utilization in education. This objective 
is fundamental to the success of other objectives of the 
Office of R&D Resources, and indeed, to the achievement 
of all NIE objectives. 

To provide the data base needed for policy analysis. 
Such a data base would (1) provide an early warning 
system to identify problems requiring policy initiatives, 
(2) assist in weighing policy alternatives by supplying 
data on the context of decision-making and the predictable 
impact of policy alternatives, and (3) provide feedback 
on the consequences of policy initiatives. 

To provide for public accountability, 
STRATEGIES 

This program is intended to establish an internal NIE capability to 
monitor the external R&D system and the operating system of education. 
Its three major components are (1) designing and assembling the necessary 
data base, (2) formulating theories and models of system dynamics, and 
(3) providing analyses and reports to internal policy-makers and NIE's 
various publics^ 

The concept of "monitoring'' is borrowed from the literature on 
social indicators. That literature was originally focussed largely 
on the identification and measurement of outcomes at a macro- 
scopic level. More recently it has come to emphasize the need to 
conceptualize models .of society or significant social sub-systems and to 
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use the models to identify the variables in all parts of the system. 
As they are concerned with the dynamic interaction between model 
elements and the measurement and understanding of change, the indicators 
must be time-series. Once the interrelationships in the model have been 
established empirically, monitoring change in the variables becomes a 
means of anticipating change in other parts of the system. As development 
of such a model is a very long-term goal, it must be approached through a 
process of successive approximations. Even so, a beginning must be made. 

There are many existing sources of data which already provide, or have 
the potential of providing, important components of a system to monitor 
educational change. These include OE's National Center for Educational 
Statistics, NSF's Science Resources Division, and the Census Bureau, as 
well as units with NIE which are developing a management information 
system and other documentation and reporting services. There is no 
desire to be redundant with these operations. The utility of their 
outputs, however, can in many cases be greatly enhanced if we can work 
with them to achieve some modifications in their data systems, such 
as obtaining data at a more disaggregated level. The cost of such a 
strategy is largely that of staff time, although in some cases we might 
obtain valuable additional data by using program funds to support 
"piggyback" studies. 

While attempting to make maximum use of intellectual and empirical 
resources in the field, we will find that there remain important kinds 
of information for which no data are currently being collected, or even 
for which no measures have yet been developed. In such cases, NIE should 
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undertake to develop appropriate measures and design appropriate new 

data-gathering efforts through a combination of intramural and contract 

research* The activities listed below will provide a starting point 

towards developing NIE's capability to monitor changes in education's 

R&D and operating systems: 

Conference on Alternative Conceptualizations of the 
Knowledge Production and Utilization System. As a 
first step towards a better understanding of the 
knowledge production and utilization process, a multi- 
disciplinary conference will be held to present papers 
summarizing the knowledge and insights which those 
disciplines can offer about this process. Leading 
experts in the sociology of science, social indicators, 
philosophy of science, economics, information theory, 
. and educational change would be among those invited. 
In addition to presenting and discussing their papers, 
the participants would be asked to work together to 
provide the outline of a possible reporting framework. 
NIE would also publish the conference papers* 

. Fact Book and Predesign Studies*. The design of the 
data system will take at least two to three years. 
In the meantime, it is important to make use of exist- 
ing data as best we can. The inventorying and analysis 
of existing data, when juxtaposed with the theory 
development conference described above, will constitute 
the first step in the design of the data system. One or 
more contracts will be let to inventory existing research 
studies. Federal statistical, systems , management informa- 
tion systems, and other sources, for information concerning 
institutions, personnel, finance, facilities, laws arid 
regulations, activities, products, and their apparent 
impact. From this information we will distill a 
parsimonious set of data for a Fact Book on Knowledge 
Production and Utilization in Education; analyze the 
data with respect to a set of policy issues determined 
by NIE; identify gaps where new data are needed; and 
make recommendations regarding the development of new 
indicators or the modification of existing data systems. 
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Design of a Sm'vey of Educational Practice and R^D 
Impact. Is American education changing rapidly, or 
is it still essentially the same as it was a genera- 
tion or two ago? One can find support for either 
premise in the research literature or the popular 
press. Schools are accused both of changing with 
glacial slowness and of chasing after every nex^7 fad. 
Baseline data are needed on the number of children 
being exposed to different kinds of educational 
programs (materials, organizational arrangements? 
and teaching-learning strategies), by level and the 
characteristics of schools, coinmunities, and pupils. 
All of this is needed as a framework for the investiga- 
tion of R&D impact. It has been pointed out that 
adoption of an innovation does not guarantee that it 
will be implemented in a manner visualized by the 
developer. But how many variations of "individualized 
instruction" are there and how does one determine whether 
a pupil is receiving it? With some innovations, in some 
situations, "prudent rejection" may be a wiser choice 
than adoption or adaption, but one must know what the 
pre-existing practice was or what alternate practice 
was preferred. The design and measurement questions 
raised by these issues are considerable. This project 
would develop operational measures of educational 
practice and design a survey which would measure the 
utilization of a range of traditional and innovative 
educational practices and R&D products. 
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VI. BUILDING THE RSD SYSTEM 

PROBLEM 

The past decade of growth in education knowledge production and 
utilization has witnessed the creation of new R&D institutions, the 
introduction of rapidly increasing numbers of new technologies, and a new 
role for the Federal Government as both a major initiator and sponsor of 
education R&D. However, that system is still in its infancy, R&D 
models often borrowed from the '^hard sciences" have not always been 
appropriate to the solution of complex social problems. New models 
more closely related to the requirements of social research must be 
invented. The new institutions and functions have yet to become part 
of an interrelated and mutually supportive endeavor. Present financing 
arrangements leave the viability of some new institutions in doubt. At 
the same time* the appropriate role of market forces and non-market- inter- 
ventions is unclear. Scientific communities which seem so vital to the 
growth of knowledge in other fields are still weak and fragmented. Strat- 
egies for increasing the responsiveness of R&D performers to the client's 
needs are underdeveloped. 

OBJECTIVES 

The objectives for a program to build the R&D system can be described 
as follows: 
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To achieve greater understanding of the structure and 
functions of the education R&D system^ including the 
identification of factors contributing to the produc- 
tivity of the system. 

To strengthen the infrastructure of institutions, 
personnel, and technology for conducting education R&D, 

• To improve NIE management of R&D^ 

To strengthen the interface between the research and 
development systems and the systems for linkage and 
utilization* 

STRATEGIES 

Because we have a very limited knowledge base from which to work, 
the conduct of further policy studies and research is essential to 
formulating long-range intervention strategies. The monitoring function 
described earlier is a major feature of this set of activities. 

In addition to long-term research and analysis, we also propose a 
series of special studies aimed at particular issues which we regard as 
crucial to NIE, These issues should be addressed in the near future, prior 
to the initiation of the comprehensive data collection strategies embodied 
in the monitoring system. A long-range program of policy studies and 
research would focus on the following areas: 

The internal operations of the knowledge production system. The 
studies will examine the manner in which various Government policies 
influence system development and would describe and analyze the current 
status of the three sub-systems: 
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The institutional sub-system of performers: what they do, 
how they do it, and how well. 

The personnel sub-system: including nature, distribution, 
and competence of personnel and the influence of training, 
selection, and recruitment strategies. 

The special facilities sub-system. 

The dynamics of the system^ s interaction with the client groups. 
The studies will examine the relationships of the schools* market struc- 
ture to the producers of R&D and the incentives in the R&D system for 
being responsive to the problems faced by schools. 

i?<SD systems in fields other than education and the education R&D 
sector in other countries. The studies will compare the nature, 
operations, and effectiveness of other systems with those of the U.S. 
education R&D system. 

NIE management strategies. The studies will identify alternative 
planning, procurement, monitoring? and evaluation strategies; assess 
their implications for attracting to NIE high quality research and 
development; and propose new management strategies. 

All of these studies are designed to gain a better appreciation of 
the dynamics of the knowledge production and utilization system and the 
problems which effect its ability to facilitate improvements in the 
nation's schools. These endeavors will identify new intervention strategies 
for improving system operations and for assessing the effects of proposed 
interventions on the total education enterprise. 
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The following short term projects are proposed for the near future* 

Policy Studies and Research 

1, Policy analysis of the roles and problems of non-profit 

and imiversity contractors. With the transition of the educational 

laboratory and R&D center program to NIE, a major shift in policy from 

"institutional support" to "program purchase" was made, severing the 

special relationship between the Government and these institutions. It 

is time to assess the policy's effect on these institutions* More 

generally, it is time to consider the impact of Government procurement 

policies on universities and other non-prof it organizations, and to 

identify other factors affecting the productivity and viability of these 

organizations. Key issues would include: 

What has been the effect of the program purchase policy on 
the effectiveness and viability of the laboratories and cen-- 
ters? 

Does the non-profit sector need strengthening, or is it 
viable and effective under present arrangements? 

What policy should NIE have on management fees? 

• What is the effect of the copyright policy in providing 
independent research funds or contributions to the capital 
structure, and is it contributing to and encouraging the 
dissemination of products in an optimal way? 

The utility of this project depends on securing the effort of experts 

who have studied these issues — possibly in fields outside of education. 

R&D — to assess available information, supplemented by limited field visits 

to some of these institutions. The expected product would be a series of 
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short-range policy recommendations for an NIE procurement policy and 
the identification of key issues for more intensive study. 

2. Policy study of alternative methods of increasing 
minority participation in education RSD, We have noted in an earlier 
section of this report that for the past ten years education of minority 
groups and the disadvantaged has been one dominant tavg^JL^ of education 
R&D, that this concern is now embodied in the legislative charter of NIE, 
that R&D personnel who are themselves members of minority groups can be 
expected to have insights into these problems not available to others, 
and that minority groups are still generally under-represented in the R&D 
work force. As it is not clear what strategies would best alleviate 
this problem, a policy study is proposed to assess the various options. 

3. Study of incentives affecting career decisions of E&D 
personnel. From a strictly supply and demand point of view, there is no 
reason to believe that education R&D should not be able to attract the 
quality and mix of persons needed for effective work. Yet it would appear 
that education R&D continues to be a low prestige field not attractive to 
our best talent. Clearly, more definitive data are needed to confirm 
these impressions. We need to know more about the incentives and 
motiviations affecting the career decisions of scientists and technicians, 
with particular reference to structural elements of the system which 
might be altered to improve the field's attractiveness and holding power. 
We propose to initiate a project to design and pre-test such a study. 
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Strengthening Institutions, Technology , 
and Personnel , 

If Workshop on research management. R&D on a scale 
sufficiently large to require sophisticated management techniques is rela- 
tively new to education, but clearly the present situation requires such 
skills and technologies. The sponsorship of a workshop on research 
management would signal NIE interest in this field, provide support for the 
growth of a related scientific community, improve the skills of personnel 
employed in management positions, and add to the technological base. 

2* Workshop and seminar series on problems in evaluation 
methodology. Evaluation has emerged as a key specialty contributing to 
the conduct of R&D in education and other social problem fields. Evalua- 
tion research is important to policy analysis, formative evaluation is 
essential to product development, and summative evaluation provides 
product validation data needed by clients and consumers in order to make 
rational adoption decisions. Pure forms of experimental design are rarely 
feasible, and new designs and methodologies are needed to cope with the 
complexities of the problems and the vagaries of working in ongoing 
social institutions. This project would sponsor a series of workshops* 
and seminars on evaluation methodology with objectives comparable to those 
for the workshop on research management , i.e., to signal NIE interest in 
the field, provide support for the growth of a scientific community, improve 
the skills of personnel working in evaluation, and add to the technological 
base. 



ERLC 



-77- 



VII. BUILDING THE LINKAGE AND 
SUPPORT SYSTEM 

The rapid growth of education knowledge production has not been 

accompanied by a parallel surge of educational reform. There seem to 

be many reasons for the apparently limited impact of education R&D. 

One problem is certainly related to the nature and quality of R&D products. 

The previous chapter suggests some ways to begin increasing the quality 

of the system's outputs. There are other problems, however, which may 

be responsive to strategies we can more rapidly initiate. These are 

generally related to difficulties which confront the operating systetn. 

For example, school staff: 

Has trouble in locating and using information 
which is relevant to its needs and which is in 
an understandable form. 

Has trouble selecting between vast numbers of 
R&D products which seem undifferentiated in 
terms of effects. 

. Needs to have access to a wide variety of alter-- 
native solutions to particular problems. 

Has trouble implementing and sustaining new 
educational approaches which are often unfamiliar 
and sometimes perceived as threatening. 

One strategy to alleviate these problems may be to develop a much 
more comprehensive and interactive system, linking together the producers 
and consumers of education knowledge and products and providing profes- 
sional support to the operating system. We have previously discussed 
several organizations and strategies which now carry out the linking 
and support functions. These included publishers, ERIC, State departments 
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of education, and teacher centers. There are many others. We propose 
here three initial program strategies designed to understand, improve, 
and strengthen this system. They are: 

. An Information Dissemination Strategy 

. A Consumer Information Strategy 

. A Froduat Delivery Strategy 

Recommendations related to external linkages are also contained in 
the chapter "Building Capacity in the Operating System." Taken together, 
the several strategies relating to linking mechanisms proposed in this 
chapter and the next are intended to provide a knowledge base that will 
eventually permit governsbental units at the local. State, and national 
levels to build together a network of linking mechanisms uniquely adapted 
to the needs of education. 

CONSUMER INFORMATION 

Education decision-makers, chief state school officers, legis- 
lators, school boards, superintendents, parents, principals, teachers, 
and children all determine the content and style of education in our 
schools. Often, they must make important decisions without much 
information about the possible effects of the various alternatives. 
Customarily, most evaluation reports have concentrated on the degree 
to which a particular product or program produces the desired outcomes 
for the target population. Occasionally they have commented on unintended 
effects as well. Only rarely have they included other information that 
has a direct bearing on the decision to adopt and the process of 
implementation — information as to community reaction, financial costs 
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of implementation, organizational consequences, effects on classroom 
discipline and vandalism, anecdotal accounts of how schools and teachers 
have coped with those problems, and so on. Yet this kind of information 
is often more important to those who must adopt and implement the product 
than is data on target group outcomes. Not only is information of this 
kind generally unavailable about Government sponsored programs, it is 
very rarely generated for innovations developed in the operating system. 
State Education Agencies, among others, are increasingly interested in 
disseminating information about good practice but have often lacked 
the resources and methodology to determine the value of innovations 
developed in the schools beyond intuitive grounds. 

Objectives 

What is needed is a program of consumer information to provide: 

Better means to identify and test promising 
practices. 

Better verification of product evaluation procedures 
and results. 

More and better information on product performance and 
the process of adoption and implementation. 

Alternative strategies for providing this information 
in an appropriate format to consumers. 

Strategies 

In order for the Consumer Information Program to provide useful and 

credible information to its intended audiences it must: 

Stimulate improved evaluations of NIE sponsored programs and 
products which generate reliable information about the effec- 
tiveness of alternatives, essential conditions for their 
implementation, and limits of their adaptability to local 
constraints. 
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Identify promising products and practices generated by the 

operating system as well as by R&D sponsors other than NIE, 
and develop a procedure for collecting and verifying similiar 
consumer information. 

Experiment with a variety of ways to effectively communi- 
cate consumer information to interested client groups- 

We propose to initiate the following experimental programs: 

1- A product verification unit- will be established in the 

Office of R&D Resources- This unit will collate and review information 

about NIE products, and verify the evaluation procedures used. This would 

include verifying the presentation of information related to adoption and 

implementation. . Appropriate courses of action will also be identified to 

determine procedures to verify non-NIE sponsored programs and products. 

2. A conference of State Agency personnel^ evaluation specialists^ 
Goveryvnent representatives ^ and others will be held to identify means of 
producing consumer information about promising school practices. The 
results of the conference will be used to develop a cooperative program 
with State Agencies and others to identify and verify school-based 
practices, to be implemented in future years. 

3. A product format study will be initiated to determine how to 
format information about promising products and practices so that it is 
useful to consumers. A contract will be let in Tl 74 to empirically 
assess the relative merits of different content and presentation styles 
for educators and school boards. 

4. A catalogue of NIE sponsored and verified products will be 
developed in FY 74 based upon initial results of the product format 
study described above. This catalogue will be designed to facilitate the 
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selection and implementation of alternative products. In the future, 
similar catalogues should be produced at least on an annual basis, 
and should include non-NIE products and school-based programs. In 
future years, other methods of communicating such information will 
also be attempted. 

5. A program of research and evaluation will be carried out 
in future years to improve the knowledge base upon which corisumer 
activities rest and to determine if the program is achieving its 
objectives. 

THE INFORMATION DISSEMINATION AND COMMUNICATION PROGRAM' 
Problem 

The foregoing analysis indicates that externally generated knowledge 
is a major resource in the process of educational reform. The results of 
research shape future research, direct the development of new educational 
solutions, and provide new insights into operating educational problems ♦ 
The experience of one school district which has designed or implemented 
an innovation can simplify the task of another school district in adapting 
or implementing a similar solution to its own problems. 

To varying degrees, members of the educational community have access 
to a substantial number of information resources. Some of these are 
focussed on specific groups of clientele, such as professional associ- 
ations and their journals, and others may deal with specific educational 
problems, such as the network of over 400 special education instructional 
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materials centers. One, the Educational Resources Information Center 
(ERIC), has undertaken — with only partial success — to provide the full 
educational community access to the comprehensive body of documentary 
English literature about education. An earlier portion of this document 
has given a brief description of ERIC and characterized it as one of the 
largest information storage and retrieval systems in the world. It is, 
of course, operated with funds from the National Institute of Education. 

Both because of ERIC's scope and its NIE sponsorship, any NIE plan 
for improved access to educational information must take into account its 
strengths and weaknesses. Hundreds of thousands of uses are made of ERIC 
resources by educators every year, yet its users make up only a fraction 
of the educational community. If one takes the perspective of a potential 
user of an information service, the reasons for under-utilization of educa- 
tional information become apparent. 

From the user's perspective, there are four major requirements of a 

responsive, serviceable information system: 

. Comprehensiveness ; few educators (or performers in other 
fields, for that matter) have the time or inclination to . 
seek information from several sources prior to making a 
decision or undertaking a course of action. 

. Relevance ; the information provided must be germane to 
the needs of the user. If one is seeking empirical 
evidence, he should not receive speculative papers; if 
one is interested in descriptions of practices in the 
school, he should not receive reports of laboratory 
experiments . 

. Utility ; the information should be as directly usable 
as possible; it should be compatible with the current 
skills and knowledge of the user. 
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. Accessibility : the user should be able to have 
preliminary access to the Information system 
locally, and preferably In his own work setting; 
the response to the Information request should 
be timely for the user. 

From this perspective, neither ERIC nor any other educational informa- 
tion system, nor all of them together, are meeting the needs of the educa- 
tional community. However, even if all these requirements are met, the 
information system will not be acceptable unless its costs are reasonable 
to the user, to the system operators, and to its sponsors. 

Objective 

The objective of the Information Dissemination and Communication 

Program can be inferred from the preceding discussion; namely. 

To provide timely access to all relevant and useful 
knowledge relating to education for the diverse 
members of the educational community, including 
teachers, administrators, school board members and 
other policy makers, researchers, and developers. 

Strategies 

Attainment of the objective requires an information system which 

performs the following functions in a cost-effective manner; 

. Acquisition of all relevant documents and other 
knowledge relating to education; 

. Screening the acquired information for authenticity, ■ 
relevance, and utility; 

Indexing the acquired information so that individuals 
addressing the information system may readily identify 
needed and relevant information; 

. Storage of information in a manner to facilitate.. . . ; 

. Retrieval of required information by System users in 
a timely and economical manner; 
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Dissernination of information in forms required by 

the user Including print, microform, computer tape, 
interpersonal communication; 

User Services for ready and timely access to the 
system and assistance in fully exploiting the 
system; 

Development Information Products such as bibliog- 
raphies, state of the art papers, and interpretations 
and syntheses of knowledge on significant educational 
topics, tailored to key user groups. 

For any system to perform these functions appreciably better than exist- 
ing services, it will be necessary to design and implement it over a 
period of several years. All of these functions are being performed 
to a greater or lesser degree by ERIC at the present time, and their 
performance is essential in providing information to educators while a 
newer, more viable system is emerging. We shall, therefore: 

1, Initially maintain the ERIC system, making such improvements 
as are possible with our current knowledge about the requirements of 
education and the state-of-the-art in information science and technology, 

2, Initiate necessary experiments, economic studies, and policy 
analyses which will provide specifications for a more viable and 
responsive information system, 

3, Design, incrementally, necessary major modifications or alter- 
natives to the emerging information system. Our initial efforts, both 
in short-term ERIC modifications and longer-range system designs will 
center around four improvements: 

, Scope of Information Covered , We shall determine ) 
whether the currently available . knowledge in ERIC 
should be broadened or narrowed, examining the 
feasibility of providing either physical access 
or references to such information as data files, 
instructional materials, and school-based information, 
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. Improved System Access , Beginning with attention 
to the list of functions above, we shall determine 
whether other functions are required, and whether 
some of those listed can be left to the attention 
of other instrumentalities. It will also be neces-- 
sary to determine the types and location of performers 
of these functions and the possibility of cooperative 
use of local. State, and Federal resources in maintain- 
ing the system developed. 

• Transformation of Information into Usable Forms . 
An expanded program for generation of problem- 
oriented information products will be begun 
immediately and further expanded in future years. 

4. Field test, adapt, and implement design alternatives for the 
improved educational information system. 

Two activities intended to provide immediate improvement will be 

initiated at once: 

, User Study . Support will be provided for a series of 
activities intended to identify current satisfaction, 
dissatisfaction, and information-using styles of 
individuals in various roles in the educational 
community. These user studies should provide an 
initial basis for making decisions about changing 
the coverage of literature and other educational . 
information. In addition, they will provide the' basis 
for initial plans for modification of local user services. 

. Information Transformation . NIE is currently in the 
process of identifying high priority topics on which 
interpretations and distillation of knowledge into 
more usable forms are required. Among the sources 
used are feed-back of stated user needs from extant 
information service centers and the results of* a 
survey of information requirements supported in 
preceding years. Using these data and staff analyses, 
NIE will issue requi-3ts for proposals for state-of-the- 
art papers and other information products on high 
priority topics. 
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Three other activities will be initiated during the current fiscal 

year or early in FY 1975, contingent upon the availability of funds: 

. Economic Analysis of ERIC , This study will address 
the issues of costs, markets, and the utility of the 
current system. A cost allocation model will be 
developed to examine the costs of each system component. 
The market analysis will answer such questions as: the 
degree to which demand for products and services depends 
upon price structure, the degree to which an information 
system can be self-supporting, and the effectiveness of 
marketing techniques in increasing the user population. 
These data will be used in future policy decisions and 
system design. 

. User Services . Awards will be made to operating informa- 
tion centers to design or implement improved or expanded 
information services to user groups. In one case, this 
may be little more than providing referral services to 
sources of technical assistance; in still others, an 
operating information service center may be supported in 
providing its services to a contiguous area which does 
not now have such services. In still another case, a 
center might be supported in the design of broader and 
more responsive information services. 

. Continued Experimentation with On-Line Computer Retrieval . 
Experience to date suggests that on-line, interactive com- 
puter search capability permits more rapid retrieval and 
generally higher user satisfaction with the results of a 
search of the ERIC file. Further experimentation with 
these techniques, both to improvr their utilization and 
to explore means of reducing their cost will be undertaken. 



The objective of this program is addressed to serving the 
manifold needs of a variety of different groups in education, each' 
having differing areas of expertise and each operating under a dif- , 
ferent set of constraints. This requires that the ultimate configu- 
ration of an effective educational information system have a wide range 
of products and service:;, tailored carefully to the diverse requirements 
of the various client groups. 
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Whatever may be the inadequacies of the current ERIC system for 
researchers, developers, and graduate students, they have the capacity 
to make use of the system (i.e., access to the products of the system 
and skill in reading and interpreting the results of research) and some 
level of incentive to seek out and use information products* Most 
individuals in the operating sector, be they policy makers, executives, 
or teachers, have relatively less capacity to seek out and exploit the 
products of an information system whose collection consist largely of 
research reports- 

We believe, therefore, that the primary, but not exclusive, focus 
of our initial efforts should be to enhance the utility of ERIC and what- 
ever system grows out of it for the educational practice and decision- 
making community. Every effort will be made, however, to ensure that 
these efforts will concurrently serve the interests of the scholarly 
community. 

These few initial activities will permit some immediate upgrading 
of the responsiveness of ERIC. In addition, however, they will provide 
the basis for a preliminary design for an improved and expanded informa- 
tion dissemination system. 

DELIVERY OF DEVELOPMENT PRODUCTS 

Improving educational practice is dependent in part upon the 
existence of a wide variety of program and product alternatives. 
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generated by both the R&D system and the operating system, which decision- 
makers can select, adapt, and implement at an acceptable cost with reason- 
able assurance of effectiveness. 

However, a variety of problems with current mechanisms for delivering 
R&D products to the operating system limits their accessibility and, con- • 
sequently, their utilization. These include a lack of continuity and 
articulation between developers and publishers, problems with products 
with low profit potential, problems of coordinating technical assistance 
with product delivery, lack of market forecasting, lack of knowledge 
about the costs and benefits of different dissemination strategies, and 
lack of incentives for developers and publishers to promote widespread 
effective utilization of their products and to revise material in the 
light of implementation effects. 

Objectives 

A series of strategies will be developed to improve education's 

product delivery system by: 

Increasing our understanding of the relative effects of a 
variety of dissemination strategies. 

Increasing the continuity between product development and 
dissemina t ion . 

. Experimenting with a variety of incentives for improving 
the delivery of high quality R&D products. 

Strategies 

We propose to initiate the following experimental activities: 
1. A managed copyright program. NIE should devise a series of 
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experimental strategies to: manage the incentives and rewards NIE 

makes available to developers/distributors; and demonstrate, create or, 

in rare instances, subsidize markets for NIE sponsored products. This 

would include: 

Demonstrating the existence of a market before inviting 
distributors, to handle the product. 

Where a market cannot be demonstrated, NIE should create 
an initial market before inviting distributors to handle 
the product. 

When an initial market can neither be demonstrated nor 
created, NIE should consider the possibility of sub- 
sidizing the limited distribution of products which may 
meet uniquely important needs. 

This program would include other experimental strategies to: provide 
incentives and rewards for cooperative arrangements between agencies that 
specialize either in development or in distribution; provide Incentives 
and rewards for agencies to extend their internal capacity to handle the 
full array of functions from product development to broad^scale distri- 
bution; and vary royalty sharing and royalty rates In contracts with 
developers/distributors in exchange for expansion of such services as 
evaluation and training. 

2. Research on the effects of alternative dissemination strategies. 
NIE should initiate a systematic series of experiments to test the effects 
of a variety of dissemination strategies to market education R&D products . 
These experiments should compare both costs of the strategy and its effects 
on the products' actual implementation. 

3. Integration of dissemination requirements with product develop- 
ment. As each NIE sponsored product approaches the field test and 

ERJC dissemination stage, the responsible program office should solicit from 
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the contractor plans for its dissemination/documentation and for a 
subsequent evaluation of the effects of those strategies. Approved 
plans should be built into the contract and funds made available to 
carry them out. The Institute should establish a mechanism for 
collecting and analyzing the reports on the costs and effects of 
various strategies to provide future guidance to program officers and 
contractors. 
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VIII. BUILDING CAPACITY IN THE OPERATING SYSTEM 

Previous chapters have discussed proposals for monitoring various 
aspects of the overall knowledge production and utilization system, for 
building the capacity of the R&D system external to the schools, and for 
developing the linkage and support system. In this chapter we address 
the problem of building capacity within the operating system. 

The chapter is divided into two parts, one relating to the State Educa- 
tion Agency (SEA) and the second concerned with building problem-solving 
capacities at the school district level. There are obvious relationships 
between the two, SEA's not only have legal responsibility for the operation 
of school systems, they also are an important resource for the development 
and support of local problem-solving activities. In addition, the diver- 
sity of State Agency functions and responsibilities makes it important to. 
summarize in a single section the variety of program recommendations 
concerning SEA's made elsewhere in this paper, 

STATE EDUCATION AGENCIES 

As indicated in Chapter III, State departments of education in the last 
decade or so have begun to exercise leadership in several aspects of the 
knowledge production and utilization process. Despite these indications 
of increased SEA capability, we know very little about the overall quality 
of SEA planning, assessment, or dissemination activities, or even about the range of 
variation that exists. To help SEA's develop capacities for linkage and 
support, we must learn more about: existing SEA programs in information 
services and dissemination; what the various states have done to identify s, 
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successful programs and promising, practices; the range of SEA activities 
to support innovation and problem-solving behavior in local districts; how 
states have been involved in product delivery systems and have provided 
local school districts with technical assistance to implement new programs; 
*the extent to which SEA's have conducted their own R&D activities to 
identify and devise solutions to state-wide educational problems; and how 
State departments of education have been reorganized to provide better 
services to local districts. 

Strategies 

A number of strategies for achieving this objective is outlined in 

other chapters of this report. Those chapters have discussed appropriate 

roles for State departments of education with regard to: 

The existing ERIC system and the development of a new educa- 
tion Information system. . 

Consumer infonnation programs, including product validation 
and the identification of promising practices. 

How sea's can assist in the development and support of 
problem-solving activities at the local school district 
level. 

Strengthening State Agency capabilities, as part of our 
study of the internal R&D system. 

Product-delivery systems, where SEA's can be involved 

in dissemination, demonstration, and technical assistance 

with regard to new educational products and practices. 

In addition to the above strategies, we propose that NIE support two 

major activities: 

(1) Surveys ccnd analytic studies of the role of State 

departments of education in the field of knowledge 
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production and utilization. These studies will 
determine present and potential capabilities of the 
various SEA's to assume leadership roles in each of 
several areas: information services, identification 
and validation of promising practices, support for local 
school district problem-solving, development of R&D 
capac.'^.ies at the State level, involvement in 
product-delivery systems, and the diffusion of exemplary 
programs. 

(2) A series of working aonferenaes^ jointly sponsored by NIE 
and the SEA's, to examine various states* experiences in 
a particular activity area (e.g., diffusion of exemplary 
programs); develop procedures for conducting activities 
that profit from the states' previous experiences and 
show promise of success; determine some systematic 
way of pilot testing the new procedures and, if the 
procedures seem successful, making them available to all 
the sea's; and establish appropriate working relation- 
ships between NIE and the states to carry out this 
process. A series of papers on the respective topics 
would be cor.n. • .?ioned as part of the planning for each 
conference. 



PROBLEM-SOLVING IN THE SCHOOLS 

Contrary to traditional models of educational change, in which 
products are generated by the formal R&D sector and then ''delivered'' to 
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the schools, practitioners can themselves be. the source of ideas and 
materials for educational improvement. In our view, the R&D community 
is one resource for innovative activity in the schools; other resources 
reside within the practitioner and lay communities. They must be 
identified, given support, and utilized more effectively. 

Because of the diversity within the operating system and the decen- 
tralized decision-making processes, reform and renewal of local school 
districts will occur only if those districts develop the capacity to be 
more analytic in their behavior, more sophisticated in the choice and use of 
resources (those generated by the R&D community as well as others), and 
better able to assess critically the effectiveness of what they are doing. 
We call this set of skills ''problem-solving behavior." 

As we indicated earlier, we currently know very little about how to 
build these capacities at the local level. The program described below 
is designed to discover, in a planned, deliberate fashion, those elements 
that contribute to the development and maintenance of problem- solving 
behavior in schools, and to test systematically the extent to which those 
elements can be combined in a few selected school districts. 
Objective 

The recommendations that follow are intended to help schools, school 
systems, and local communities build an organizational capability for 
problem-solving behavior. We believe that such capability will lead 
to: 

. More effective use of the products of the external R&D 
resource system. 
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Increased and better articulated demand from the schools 
for more appropriate R&D products. 

Increased ability at the local level to utilize local 
resources effectively for reform and renewal in a manner 
responsive to locally perceived problems. 

Strategies 

Our major strategy is a. program of directed experimental studies to 
develop, test, and disseminate approaches for building problem-solving 
capacities in schools. The experimental program is designed to examine • 
the effects of these approaches within the four contexts discussed earlier: 

• The teacher leve l, where the target population consists of 
individual or small groups of teachers and the strategies 
are directed at changes in individual classrooms. 

. The individual school level , where an entire school or mini- 
school is the unit of analysis and change strategies 
consequently must take into account organizational and 
inter'^personal dynamics. 

The school district level , where the emphasis is on building 
problem-solving capacities within an entire school system 
or sub-system. The focus here is generally on the redefined 
roles of central office administrative and support staffs. 

. The school and community level , where citizens outside the 
formal school system :stimulate and help to sustain a problem- 
solving orientation in the operation of the community's 
schools. 

The recommendations are grouped into four contexts or levels of 
educational change because there appears to be little consensus regarding 
the most appropriate or cost-effective leverage points for improving the 
educational system. Many observers have noted that the number of inter- 
connected elements in an educational system is so complex that there is an 
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ever-increasing series of options which can be selected as entry-points for 
change at each level, especially with respect to the capacity-building 
orientation we are suggesting. We must also explore more systematically 
the relationship between levels. 

Consequently, the program recommendations below are designed to 
accumulate information — from surveys, case studies of existing programs, 
and the initiation and replication of promising alternatives — to assess 
the effects of the proposed activities at each level. As we begin to 
understand the effects and potential of proposed alternatives at each level, 
we will attempt to integrate the most promising components at a few 
carefully selected sites or ''assembly" districts. We will identify at 
least three school systems of contrasting characteristics where, begin- 
ning in the third year of the program, we will have an opportunity to test 
combinations of programs, working*- toward some optimal integration of 
components for each level. 

Throughout the development of this program we will be concerned 
with the policy implications of our findings. If our research indicates, 
for example, that a particular kind of teacher center is successful in 
promoting problem-solving among teachers in urban elementarj^ schools, we 
must address the question of how to utilize these findings: What is the. 
optimum number of clients this type of center can serve? What are the 
policy options for funding large numbers of these centers? What roles 
might sea's play in the process? What are the policy implications for 
the Federal Government in terms of new legislation, cooperative efforts 
between NIE and the U.S. Office of Education, or new relationships vis-a- 
^l^Q State and local education agencies? 
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Program Elements 

.The program begins with four parallel sets of activities. For each 
of the four levels — teacher, school, school district, and school/community — 
there will be a program consisting of the following elements: 

(1) Surveys , Syntheses , and Analyses . In the first year, surveys will 
be conducted to determine the extent and nature of existing variation within 
each of the four levels. The initial surveys will include historical 
reviews as well as current practices, and will identify situations to be 
included in the series of longitudinal case studies described below. After 
the first year or two, the surveys will be more analytic in emphasis, 
integrating findings from the ongoing case studies, and analyzing the policy 
implications of what is being learned. 

(2) Three-Year, Longitudinal Case Studies . For each level, a prime 
contractor would be chosen with broad experience in observation and 
interview techniques to conduct longitudinal case studies. Although the 
methodological emphasis will be placed on anthropological case study 
techniques, we will expect the contractor to conduct sample surveys and 
assessments of student outcomes where appropriate. 

Each series of case studies will consist of about six sites. Series 
t case studies will be initiated prior to the availability of final survey 
results. We will select programs at each level that we have identified 
from initial field work as having promise in promoting problem-solving 
behavior. The survey findings and experience with studying these six sites 
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will help to identify sites for the Series 2 case studies. Among the 
programs included in this second series will be new interventions that ^ 
have been developed specifically for this program. The sites in the 
Series 3 ease studies will consist of two kinds of programs; new inter- 
ventions designed in response to what has been learned from the analyses 
and prior case studies; and deliberate attempts to replicate programs that 
have demonstrated success, 

(3) Program Development , Funds will be provided to selected sites 
to initiate, replicate, or strengthen specified activities that are related 
to our problem-solving framework. For example, grants might be given to 
help sites develop training or technical assistance programs, or to initiate 
a communications and support network among several agencies. 

In addition to support for the field sites, the development program 
will include an intensive design phase for new interventions to be tested 
in the Series 2 and Series 3 case studies. This design activity will 
draw on projects currently being supported by other NIE units as well as 
developments in the educational field generally. In the former category, 
for instance, there are a number of NIE contractors currently working on 
projects related to our definition of problem-solving. We will review 
those projects to determine whether there are program elements that can 
be further elaborated and tested within our problem-solving framework. 

We also will be responsive to more general developments that have 
implications for proposed interventions. At .the teacher level, for 
example, many colleges and universities are showing increased interest in 
in-service teacher education, due primarily to the reduced need for new 
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teachers and rthe resulting financial difficulties for schools of education. 
In addition, the competency-based teacher education movement — currently 
affecting certification requirements in more than half the states — presents 
opportunities for new program designs that we can test in Series 2 and 3. 
Among the interventions that might be developed are: 

Clinical training and development centers which 
use a school setting for instructional research, 
curriculum development, and in-service teacher 
education. 

Portal schools which use competency-based procedures 
to select and train new teachers for urban schools, 
and then provide in-service support for them during 
their initial years of teaching. 

Programs of joint faculty appointments and other 
incentives for acquiring the time and commitment 
of college and university faculty members in 
providing sustained research, development, and 
training help to teachers in their school settings. 

(4) Technical. Assistance and Policy Analysis . The primary intent of 

of the surveys and case studies is to provide information for consumers 

and decision-makers — for teachers' associations considering whether to 

negotiate for a teacher center; for a group of parents engaged in the 

design of an alternative school; for a school superintendent in the 

process of reorganizing his central office; for a principal creating a 

new governance system for his school; for State legislators drafting a 

teacher certification bill; or for Federal policy-makers considering how 

USOE and NIE might relate to State and local education programs. 
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At each stage of this program's development, and for each of the 
four levels, we will concentrate on making relevant findings available 
in appropriate ways for various audiences. The evaluation contractors 
will provide technical assistance, and a major element of their charge 
will be to develop ways for transforming what they are learning from 
their evaluations into print, audio-visual formats, or person-to-person 
communications arrangements so that others may profit from the intensive 
studies being undertaken. As indicated earlier, contractors engaged in 
the ongoing syntheses also will have to address the policy implications 
of the findings at each stage of the program. 

Integration and Utilization of Findings 

During the third year, we will add another dimension to the program, 
on. which begins to integrate findings from the four parallel lines of 
activity. In our "assembly districts," we will experiment with combina- 
tions of program elements which were found to promote problem-solving 
behavior at each of the four levels. Our aim will be to discover how 
these elements can be implemented and adapted in combinations and/or 
sequences that are consistent and mutually-reinforcing. Although we 
believe that findings from any one of the four levels can be extremely 
useful for many consumers and decision-makers, we are most interested 
in the interactions among the components. A community has an 
increased likelihood of sustaining a continual process of reform and 
renewal j we feel, if it supports problem-solving activities in 
appropriate combinations at multiple levels, from individual classrooms 
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to the wider conununity outside the schools • Our experimentation in 
the assembly districts is designed to help us discover how best to 
integrate these elements. 

In the chart below we summarize the general research plan for the 
next five years. The reader should note that the sequence below is 
for each of the four levels. In addition to the syntheses and cross- 
level experimentation in the assembly districts in later years of 
the program, there 1« a separate series of surveys, case studies, 
program development, policy analysis, and technical assistance activities 
for each level. 

We do not anticipate that this program will "terminate" in 1979. 
We have projected a five-year timetable primarily for planning and 
budgeting purposes, NIE has made a major commitment to building problem- 
solving capacities at the local level and we anticipate that this issue 
will 'continue to be a focus for further experimentation and development. 
Our aim, however, is to make this issue a concern of other Government 
agencies as well, at the State, loca], and national levels. As a result 
of our technical assistance and policy analysis activities, we intend 
to engage such groups as LEA's, local community agencies, teachers unions 
and associations. State departments of education, and the U,S, Office of 
Education in discussions of how this work might assist them in their 
own activities , 
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(Fiscal Years) 



74 75 76 77 78 79 



Surveys, Syntheses, and 

Policy Analyses x x x x 

Case Study Series I 

Program Development x x x 

Documentation/Evaluation x x x 

Case Study Series 2 

Program Development x x 

' Documentation/Evaluation x x 



X 
X 



Case Study Series 3 
Program Development 

and Replication x x x 

Documentation/Evaluation x x x 

Assembly Districts x x x 

Dissemination and 

Implementation Assistance x x X x x 



The Four Levels; Issues and Initiatives 

Much of the discussion to this point has been general and concerned 
with sequence and methodology; the four levels have been mentioned briefly, 
but little has been said about their substantive orientation. Although it 
is not possible in this paper to provide much substantive discussion about 
the content of the four levels, the following sections describe 
generally the surveys, case studies, and development programs that might 
be designed for each level. 

Teacher Level . The surveys in the first year or two will concentrate 
on teacher support agencies (often called "intermediaries"). These might 
include teacher centers, advisories, and extension agents which provide 
individual teachers with the knowledge, resources, and motivation necessary 
to become more effective in: diagnosing their classroom problems, locating 
information and materials to address those problems, adapting and applying 
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those resources in the classroom, and evaluating the effects of their 
teaching. A major aim of the surveys will be to construct a typology 
or analytic framework that encompasses the broad range of organizations 
and strategies designed to provide professional support to teachers. The 
typology and surveys as continuously refined will form the basis for 
identifying cases for Series 2 and 3 longitudinal studies; for research 
and analysis to compare variables across sites; for stimulating new 
intermediary arrangements; and for replicating promising models. 

For the initial series of case studies, we propose to select one 
class of professional support strategies — teacher center/advisory 
arrangements — which appear to be promising means of facilitating problem- 
solving behaviors in teachers. While we have also identified other 
useful classes of support (e*g., extension agents, technical assistance 
programs in SEA's, Follow-Through delivery strategies), we believe that 
the teacher center movement lias sufficient currency, momentum, and 
promise that it is useful to analyze this potential as rapidly as possible. 

Sites for Series 2 and Series 3 case studies will be selected on the 
basis of survey data and Series 1 case study results which will indicate 
the relative importance of several variables (organization, financing, 
leadership style, ideological orientation, etc.). We also will be 
designing new interventions and reviewing other NIE-supported programs 
to determine whether they relate to problem-solving at the teacher level. 
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We are also planning to provide support for field sites participating 

in the case study series both to strengthen their capacity to affect 

client groups and to introduce variations which can then be compared 

across sites under a variety of conditions. For example: 

Funds would be provided to a limited number of teacher 
center sites to add on new services. For example, 
teacher centers in rural areas can be funded to add 
on advisory and outreach services similar to those 
which already exist in many urban sites. 

Funds would be provided to organizations to increase 
their interaction with each other so that the effects 
of networking strategies on the individual capacities 
of sites can be analyzed. This could include develop- 
ing communication mechanisms to share information and 
experience, sponsoring seminars or conferences on 
areas of mutual interest, joint staff training, and 
conducting research across sites. 

Funds would be provided to a limited number of sites 
to enable them to render technical assistance to 
other agencies that want to implement teacher support 
interventions. Funds could be provided for training 
stafJ: of new sites, exchanging personnel* and providing 
ongoing advisory assistance. 

School Level . The surveys will identify and assess strategies that 
have been used to effect whole-school change. We are interested in the 
recent history of externally-generated efforts (e.g., COPED, EPDA 
Differentiated Staffing Programs) to stimulate organizational change 
in schools, and wi*at -key variables can be identified to guide the 
development of our own program. 

For the case studies we will follow the same format outlined above 
for the teacher level. Among the existing programs to be studied in 
Series 1, we might look for: schools with internal research staffs; 
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examples In which staffs have been trained In preparation for the 
opening of new schools; and schools whose faculties have participated 
In Intensive organization development programs. 

In the design of Interventions for Series 2 and Series 3 case 
studies we will use the findings from the surveys and Initial case 
studies. In addition, we will review programs supported elsewhere In the 
Institute — such as the Multi-Unit School program of the Wisconsin R&D 
Center or the Strategies for Organizational Change program at the 
University of Oregon R&D Center — to determine their appropriateness for 
our program and as a basis for Initiating design studies for alternative 
strategies. 

School District Level . The surveys In the first year or two will 
be designed to determine the existing variation In function and 
effectiveness of school district research/ planning/evaluation offices. 
We are Interested In the resources allocated by school districts for 
these functions and how these resources have contributed to problem- 
solving behavior at a school district level. 

Series 1 case studies will examine a small number of school districts 
Identified as having elements of problem-solving capacity already In 
existence. For example, we might look for school districts that conduct 
organization development activities, those that have a multi-faceted 
research and evaluation office, or those that sponsor extensive develop- 
ment and training programs* 
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Our design of new programs to be tested and analyzed In the 

Series 2 and 3 case studies will be influenced to a significant extent 
by what we learn from the surveys and Series 1 case studies. In 
addition, however, there are a number of ongoing NIE sponsored programs 
that relate to change at the school district level. The following 
are among the programs we will review in determining specific inter- 
ventions to be developed and tested: 

. Administering for Change (Research for Better Schools) 
Responsiveness of Public Schools (Oregon R&D Center) 
Evaluation Technologies Program (UCLA R&D Center) 
Educational Management Program (Far West Laboratory) 
School/Community Level . As with the other three levels, the 
initial surveys for the school/coinmunity level are intended to learn 
from recent or existing programs. With regard to the school/community 
level, where we are interested in the interaction of parents and non- 
professionals with their schools, the following questions might be 
explored in the surveys and analyses: Where have non-professionals been 
involved in defining a community's educational goals and in monitoring 
the performance cf schools? What mechanisms have proven useful in 
involving parents and community in schools? What are some successful 
techniques for providing information about a school's performance to 
parents? To what extent has media been used to promote debate in 
communities about educational goals? 
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The initial case studies will examine a few settings in which non- 
professionals have had a significant role in stimulating a problem- 
sol^ 'ng orientation for their community's schools. Examples of the 
situations we have in mind are: communities where non-professionals have 
had important roles in the definition of educational goals; communities 
in which parents have been involved in an ongoing way in influencing 
the program/staff /organization/budget of their neighborhood school; 
and communities where information services have provided parents with 
support in affecting changes. 

As with the other levels, the design of interventions for the 
. Series 2 and 3 case studies will be influenced by several factors: 

findings from the surveys and initial case studies; the appropriateness 
of elements from other NIE-supported projects; and unanticipated develop- 
ments in education generally. The purpose of our sequenced, deliberate 
strategy is to accommodate these various influences and to permit us to 
profit from what we learn at each preceding stage of the design. Among 
the NIE supported projects we will review as we design new interventions 
for the school/community level are: 

. The School-Community Input Team as a Social Intervention 
(Palo Alto Unified School District) 

. Reviewing Home-School Linkage (Far West Laboratory) 

. National Sampling Survey of Community Control of 
Public Elementary and Secondary Schools (National 
Opinion Research Center) 

.A Laboratory Investigation of the Goals of Secondary 
Education as Perceived by Educational Consumers (Huma- 
nistic Designs Corp.) 

. Rural Education Program (Northwest Regional Laboratory) 
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SUMMARY 

Overall, the strategy for the experimental program contains the 

following interrelated components, each building on the data and 

experience acquired by the previous elements: 

. Surveys of the natural variation among existing insti- 
tutional arrangements that have potential to stimulate 
and support problem-solving behavior. These will be 
organized along the four levels or contexts of change 
described earlier: teachers, schools, school districtst 
and school/communities. 

Intensive case studies of selected interventions that 
have potential to support problem-solving behavior at 
the four levels. Series 1 will study existing programs. 
Series 2 will combine existing programs and new inter- 
ventions, and Series 3 will focus on additional inter- 
ventions and deliberate attempts at replication. 

. Technical assistance to agencies that wish to adapt 
and implement individual interventions developed by the 
program. Responsibility for providing this assistance 
will be part of the contracts to conduct the case 
studies and will also be assumed by the field sites 
being studied and by NIE program officers. 

. Assembly of successful interventions from each of the 
four levels in a small number of school districts to 
study their interaction. The intent is to determine 
how the various components can be adapted in order both 
to complement each other and, in their interactions, to 
meet local conditions. 

Finally, technical assistance to school districts that want 
to build their problem-solving capacity by systematically 
implementing related, mutually-supportive interventions. 
This assistance would be based on the experience with 
the assembly districts, where the combination of elements 
had been tested and analyzed. 
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A series of policy analyses will be prepared at each stage of this 
program to relate appropriate findings to policy options and the activities 
of other Government agencies. These studies will be intended to acquaint 
policy-makers at various Governmental levels with the implications of this 
research for their respective areas of responsibility. 
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IX, FIELD INITIATED STUDIES 

The directed program described in the preceding chapters will reflect 
a specific viewpoint about production, dissemination, and utilization of 
scientific knowledge in education. We recognize, however, that there are 
competing perspectives on these processes* Accordingly, we believe that 
the overall program of the Institute should allow Such perspectives to 
be considered in light of the significance and scientific merit of proposals 
which espouse them. In the process, it is possible that results of such 
research may refine or modify the assumptions and emphases of the 
directed program. 

To accomplish this, we propose support of research on the processes 
of educational change, rather than support of actual dissemination or 
demonstration activities. We believe that stimulation of research in 
this area will contribute badly needed fundamental and applied knowledge 
about highly significant but poorly understood phenomena in education. 

This area of inquiry could provide knowledge on the full range of 
phenomena, performers, and settings involved in the improvement of educa- 
tion through production of research results, systematic development of 
alternative solutions to educational problems, dissemination of the results 
of R&D throughout the education community, and the adoption and imple- 
mentation of the results of research and development for improved education. 
The scope should include, but not be limited to, studies of: 
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The existing or potential education R&D system, including 
institutional and organizational settings in which educa- 
tion R6rD is or might be performed; personnel for education 
R&D; incentives and impediments to participation in educa- 
tion R&D; and factors influencing the quality of education 
R&D. 

Improved means of maintaining and providing access to the 
documentary knowledge about education, including the 
acquisition, processing, storage, retrieval, and 
dissemination of documentary knowledge. 

The spread of the results of research and development through 
the education system, including tracer studies of the spread 
and adoption of specific innovations, applications of market 
research techniques to educational change, and the effects 
of differing dissemination strategies. 

The effects of alternative means of linking R&D and educa- 
tional practice in efforts to improve the operation of 
educational institutions. 

Innovative and problem-solving behavior in educational 
practice settings, including studies of incentives and 
predictors of educational change, organizational 
behavior in education institutions, characteristics of 
innovative educators and institutions, and the implemen- 
tation of innovations in education institutions. 

Together, the directed program and the field-initiated studies should, 

over time, provide the basis for development of a coherent, integrated 

Government policy for knowledge production and utilization in education. 

This policy development will always be in process and should continue to 

be a cooperative endeavor among all the individuals and organizations that 
share a commitment to the improvement of American education. 
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